Note: This is test shard 1 of 8.
[==========] Running 8 tests from 6 test suites.
[----------] Global test environment set-up.
[----------] 1 test from AdminCliTest
[ RUN ] AdminCliTest.RebalancerReportOnly
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240502 07:58:06.548458 15878 test_util.cc:268] Using random seed: -157360683
I20240502 07:58:06.555403 15878 ts_itest-base.cc:112] Starting cluster with:
I20240502 07:58:06.555467 15878 ts_itest-base.cc:113] --------------
I20240502 07:58:06.555482 15878 ts_itest-base.cc:114] 5 tablet servers
I20240502 07:58:06.555502 15878 ts_itest-base.cc:115] 3 replicas per TS
I20240502 07:58:06.555518 15878 ts_itest-base.cc:116] --------------
2024-05-02T07:58:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T07:58:06Z Disabled control of system clock
I20240502 07:58:06.565073 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.129.190:40863
--webserver_interface=127.15.129.190
--webserver_port=0
--builtin_ntp_servers=127.15.129.148:42717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.129.190:40863 with env {}
W20240502 07:58:06.675426 15890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:06.675649 15890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:06.675689 15890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:06.679158 15890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 07:58:06.679226 15890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:06.679250 15890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 07:58:06.679272 15890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 07:58:06.683276 15890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:42717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.129.190:40863
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.129.190:40863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.15.129.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.15890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:06.684212 15890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:06.685408 15890 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:06.694835 15898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:06.696234 15895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:06.697350 15896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:06.698376 15890 server_base.cc:1034] running on GCE node
I20240502 07:58:06.698982 15890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:06.700023 15890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:06.701206 15890 hybrid_clock.cc:648] HybridClock initialized: now 1714636686701193 us; error 28 us; skew 500 ppm
I20240502 07:58:06.703276 15890 webserver.cc:460] Webserver started at http://127.15.129.190:33653/ using document root <none> and password file <none>
I20240502 07:58:06.703864 15890 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:06.703929 15890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:06.704128 15890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:06.706054 15890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "74778355d3914554a21a5a17d75c312d"
format_stamp: "Formatted at 2024-05-02 07:58:06 on dist-test-slave-0hx2"
I20240502 07:58:06.707175 15890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "74778355d3914554a21a5a17d75c312d"
format_stamp: "Formatted at 2024-05-02 07:58:06 on dist-test-slave-0hx2"
I20240502 07:58:06.710665 15890 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:58:06.713147 15904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:06.714211 15890 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:58:06.714339 15890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
uuid: "74778355d3914554a21a5a17d75c312d"
format_stamp: "Formatted at 2024-05-02 07:58:06 on dist-test-slave-0hx2"
I20240502 07:58:06.714426 15890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:06.733826 15890 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:06.734480 15890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:06.734638 15890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:06.741253 15956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.190:40863 every 8 connection(s)
I20240502 07:58:06.741261 15890 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.190:40863
I20240502 07:58:06.742285 15890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 07:58:06.744752 15957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:06.749547 15957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d: Bootstrap starting.
I20240502 07:58:06.750968 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 15890
I20240502 07:58:06.751104 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 07:58:06.752168 15957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:06.752993 15957 log.cc:824] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:06.755352 15957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d: No bootstrap required, opened a new log
I20240502 07:58:06.760073 15957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74778355d3914554a21a5a17d75c312d" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 40863 } }
I20240502 07:58:06.760299 15957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:06.760385 15957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74778355d3914554a21a5a17d75c312d, State: Initialized, Role: FOLLOWER
I20240502 07:58:06.760921 15957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74778355d3914554a21a5a17d75c312d" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 40863 } }
I20240502 07:58:06.761158 15957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:06.761296 15957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:06.761473 15957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:06.782079 15957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74778355d3914554a21a5a17d75c312d" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 40863 } }
I20240502 07:58:06.782568 15957 leader_election.cc:302] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74778355d3914554a21a5a17d75c312d; no voters:
I20240502 07:58:06.782979 15957 leader_election.cc:288] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:06.783118 15962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:06.783334 15962 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d [term 1 LEADER]: Becoming Leader. State: Replica: 74778355d3914554a21a5a17d75c312d, State: Running, Role: LEADER
I20240502 07:58:06.783752 15962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74778355d3914554a21a5a17d75c312d" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 40863 } }
I20240502 07:58:06.784253 15957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d [sys.catalog]: configured and running, proceeding with master startup.
I20240502 07:58:06.785435 15963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74778355d3914554a21a5a17d75c312d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74778355d3914554a21a5a17d75c312d" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 40863 } } }
I20240502 07:58:06.785580 15963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d [sys.catalog]: This master's current role is: LEADER
I20240502 07:58:06.785981 15964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74778355d3914554a21a5a17d75c312d. Latest consensus state: current_term: 1 leader_uuid: "74778355d3914554a21a5a17d75c312d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74778355d3914554a21a5a17d75c312d" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 40863 } } }
I20240502 07:58:06.786183 15964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d [sys.catalog]: This master's current role is: LEADER
W20240502 07:58:06.788990 15977 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240502 07:58:06.789052 15977 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240502 07:58:06.789129 15971 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 07:58:06.789847 15971 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 07:58:06.794762 15971 catalog_manager.cc:1347] Generated new cluster ID: 06d1430352d94d5a84ac16c228ee71e4
I20240502 07:58:06.794829 15971 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 07:58:06.801602 15971 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 07:58:06.802776 15971 catalog_manager.cc:1504] Loading token signing keys...
I20240502 07:58:06.813450 15971 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 74778355d3914554a21a5a17d75c312d: Generated new TSK 0
I20240502 07:58:06.814333 15971 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 07:58:06.819437 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.129:0
--local_ip_for_outbound_sockets=127.15.129.129
--webserver_interface=127.15.129.129
--webserver_port=0
--tserver_master_addrs=127.15.129.190:40863
--builtin_ntp_servers=127.15.129.148:42717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20240502 07:58:06.935604 15981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:06.935844 15981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:06.935920 15981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:06.939091 15981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:06.939237 15981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.129
I20240502 07:58:06.943636 15981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:42717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.15.129.129
--webserver_port=0
--tserver_master_addrs=127.15.129.190:40863
--never_fsync=true
--heap_profile_path=/tmp/kudu.15981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.129
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:06.944545 15981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:06.945740 15981 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:06.955222 15986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:06.956552 15987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:06.957909 15989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:06.958904 15981 server_base.cc:1034] running on GCE node
I20240502 07:58:06.959399 15981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:06.960029 15981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:06.961196 15981 hybrid_clock.cc:648] HybridClock initialized: now 1714636686961187 us; error 27 us; skew 500 ppm
I20240502 07:58:06.963075 15981 webserver.cc:460] Webserver started at http://127.15.129.129:35429/ using document root <none> and password file <none>
I20240502 07:58:06.963613 15981 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:06.963682 15981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:06.963855 15981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:06.965615 15981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "f881831625a4439e88fbc46bb7582205"
format_stamp: "Formatted at 2024-05-02 07:58:06 on dist-test-slave-0hx2"
I20240502 07:58:06.966071 15981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "f881831625a4439e88fbc46bb7582205"
format_stamp: "Formatted at 2024-05-02 07:58:06 on dist-test-slave-0hx2"
I20240502 07:58:06.969290 15981 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20240502 07:58:06.971529 15995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:06.972602 15981 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:06.972718 15981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "f881831625a4439e88fbc46bb7582205"
format_stamp: "Formatted at 2024-05-02 07:58:06 on dist-test-slave-0hx2"
I20240502 07:58:06.972807 15981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:06.991014 15981 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:06.991655 15981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:06.991815 15981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:06.992348 15981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:06.993235 15981 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:06.993284 15981 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:06.993378 15981 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:06.993408 15981 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:07.001514 15981 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.129:39041
I20240502 07:58:07.001550 16108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.129:39041 every 8 connection(s)
I20240502 07:58:07.002429 15981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 07:58:07.005744 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 15981
I20240502 07:58:07.005843 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 07:58:07.007902 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.130:0
--local_ip_for_outbound_sockets=127.15.129.130
--webserver_interface=127.15.129.130
--webserver_port=0
--tserver_master_addrs=127.15.129.190:40863
--builtin_ntp_servers=127.15.129.148:42717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240502 07:58:07.012976 16109 heartbeater.cc:347] Connected to a master server at 127.15.129.190:40863
I20240502 07:58:07.013216 16109 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:07.013867 16109 heartbeater.cc:511] Master 127.15.129.190:40863 requested a full tablet report, sending...
I20240502 07:58:07.015573 15921 ts_manager.cc:194] Registered new tserver with Master: f881831625a4439e88fbc46bb7582205 (127.15.129.129:39041)
I20240502 07:58:07.017386 15921 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.129:54103
W20240502 07:58:07.115775 16112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:07.116008 16112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:07.116057 16112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:07.119167 16112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:07.119318 16112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.130
I20240502 07:58:07.122956 16112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:42717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.15.129.130
--webserver_port=0
--tserver_master_addrs=127.15.129.190:40863
--never_fsync=true
--heap_profile_path=/tmp/kudu.16112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.130
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:07.123876 16112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:07.125013 16112 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:07.134364 16117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:07.135291 16118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:07.136334 16120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:07.138140 16112 server_base.cc:1034] running on GCE node
I20240502 07:58:07.138602 16112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:07.139220 16112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:07.142488 16112 hybrid_clock.cc:648] HybridClock initialized: now 1714636687142459 us; error 42 us; skew 500 ppm
I20240502 07:58:07.144428 16112 webserver.cc:460] Webserver started at http://127.15.129.130:45535/ using document root <none> and password file <none>
I20240502 07:58:07.144963 16112 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:07.145017 16112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:07.145169 16112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:07.146823 16112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "a9e5dfa96d1f4b7d89aea4ce3575439c"
format_stamp: "Formatted at 2024-05-02 07:58:07 on dist-test-slave-0hx2"
I20240502 07:58:07.147250 16112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "a9e5dfa96d1f4b7d89aea4ce3575439c"
format_stamp: "Formatted at 2024-05-02 07:58:07 on dist-test-slave-0hx2"
I20240502 07:58:07.150386 16112 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20240502 07:58:07.152432 16126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:07.153477 16112 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:07.153579 16112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "a9e5dfa96d1f4b7d89aea4ce3575439c"
format_stamp: "Formatted at 2024-05-02 07:58:07 on dist-test-slave-0hx2"
I20240502 07:58:07.153676 16112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:07.166483 16112 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:07.167107 16112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:07.167265 16112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:07.167775 16112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:07.168632 16112 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:07.168678 16112 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:07.168735 16112 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:07.168771 16112 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:07.177481 16112 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.130:38981
I20240502 07:58:07.177518 16239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.130:38981 every 8 connection(s)
I20240502 07:58:07.178361 16112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 07:58:07.184279 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 16112
I20240502 07:58:07.184401 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 07:58:07.187060 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.131:0
--local_ip_for_outbound_sockets=127.15.129.131
--webserver_interface=127.15.129.131
--webserver_port=0
--tserver_master_addrs=127.15.129.190:40863
--builtin_ntp_servers=127.15.129.148:42717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240502 07:58:07.187404 16240 heartbeater.cc:347] Connected to a master server at 127.15.129.190:40863
I20240502 07:58:07.187605 16240 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:07.188113 16240 heartbeater.cc:511] Master 127.15.129.190:40863 requested a full tablet report, sending...
I20240502 07:58:07.189090 15921 ts_manager.cc:194] Registered new tserver with Master: a9e5dfa96d1f4b7d89aea4ce3575439c (127.15.129.130:38981)
I20240502 07:58:07.189693 15921 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.130:57061
W20240502 07:58:07.294436 16243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:07.294659 16243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:07.294729 16243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:07.298074 16243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:07.298211 16243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.131
I20240502 07:58:07.301918 16243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:42717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.15.129.131
--webserver_port=0
--tserver_master_addrs=127.15.129.190:40863
--never_fsync=true
--heap_profile_path=/tmp/kudu.16243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.131
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:07.302806 16243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:07.303912 16243 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:07.313424 16251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:07.314546 16248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:07.315680 16249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:07.316982 16243 server_base.cc:1034] running on GCE node
I20240502 07:58:07.317478 16243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:07.318138 16243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:07.319295 16243 hybrid_clock.cc:648] HybridClock initialized: now 1714636687319280 us; error 31 us; skew 500 ppm
I20240502 07:58:07.321182 16243 webserver.cc:460] Webserver started at http://127.15.129.131:34139/ using document root <none> and password file <none>
I20240502 07:58:07.321841 16243 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:07.321905 16243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:07.322081 16243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:07.346539 16243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "456c02424ee24e4987d07088be526ea4"
format_stamp: "Formatted at 2024-05-02 07:58:07 on dist-test-slave-0hx2"
I20240502 07:58:07.347060 16243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "456c02424ee24e4987d07088be526ea4"
format_stamp: "Formatted at 2024-05-02 07:58:07 on dist-test-slave-0hx2"
I20240502 07:58:07.350586 16243 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:58:07.352838 16257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:07.353945 16243 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:58:07.354079 16243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "456c02424ee24e4987d07088be526ea4"
format_stamp: "Formatted at 2024-05-02 07:58:07 on dist-test-slave-0hx2"
I20240502 07:58:07.354194 16243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:07.375042 16243 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:07.375726 16243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:07.375886 16243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:07.376406 16243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:07.377305 16243 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:07.377385 16243 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:07.377429 16243 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:07.377449 16243 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:07.386647 16243 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.131:44093
I20240502 07:58:07.386754 16370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.131:44093 every 8 connection(s)
I20240502 07:58:07.387578 16243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 07:58:07.395195 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 16243
I20240502 07:58:07.395309 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 07:58:07.396749 16371 heartbeater.cc:347] Connected to a master server at 127.15.129.190:40863
I20240502 07:58:07.397004 16371 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:07.397589 16371 heartbeater.cc:511] Master 127.15.129.190:40863 requested a full tablet report, sending...
I20240502 07:58:07.397604 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.132:0
--local_ip_for_outbound_sockets=127.15.129.132
--webserver_interface=127.15.129.132
--webserver_port=0
--tserver_master_addrs=127.15.129.190:40863
--builtin_ntp_servers=127.15.129.148:42717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240502 07:58:07.398624 15920 ts_manager.cc:194] Registered new tserver with Master: 456c02424ee24e4987d07088be526ea4 (127.15.129.131:44093)
I20240502 07:58:07.399437 15920 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.131:53801
W20240502 07:58:07.505828 16374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:07.506064 16374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:07.506134 16374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:07.509181 16374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:07.509346 16374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.132
I20240502 07:58:07.512962 16374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:42717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.15.129.132
--webserver_port=0
--tserver_master_addrs=127.15.129.190:40863
--never_fsync=true
--heap_profile_path=/tmp/kudu.16374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.132
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:07.513870 16374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:07.514962 16374 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:07.524375 16380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:07.525348 16379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:07.526345 16382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:07.527863 16374 server_base.cc:1034] running on GCE node
I20240502 07:58:07.528388 16374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:07.528956 16374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:07.530107 16374 hybrid_clock.cc:648] HybridClock initialized: now 1714636687530082 us; error 39 us; skew 500 ppm
I20240502 07:58:07.531932 16374 webserver.cc:460] Webserver started at http://127.15.129.132:41695/ using document root <none> and password file <none>
I20240502 07:58:07.532466 16374 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:07.532521 16374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:07.532706 16374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:07.534412 16374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "48ab489bc1f54a65bd73c6c3cad9b246"
format_stamp: "Formatted at 2024-05-02 07:58:07 on dist-test-slave-0hx2"
I20240502 07:58:07.534890 16374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "48ab489bc1f54a65bd73c6c3cad9b246"
format_stamp: "Formatted at 2024-05-02 07:58:07 on dist-test-slave-0hx2"
I20240502 07:58:07.538158 16374 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:58:07.540403 16388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:07.541579 16374 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:58:07.541692 16374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "48ab489bc1f54a65bd73c6c3cad9b246"
format_stamp: "Formatted at 2024-05-02 07:58:07 on dist-test-slave-0hx2"
I20240502 07:58:07.541779 16374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:07.567791 16374 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:07.568498 16374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:07.568668 16374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:07.569226 16374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:07.570140 16374 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:07.570190 16374 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:07.570255 16374 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:07.570281 16374 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:07.579110 16374 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.132:44329
I20240502 07:58:07.579149 16501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.132:44329 every 8 connection(s)
I20240502 07:58:07.580049 16374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 07:58:07.583765 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 16374
I20240502 07:58:07.583889 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 07:58:07.586138 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.133:0
--local_ip_for_outbound_sockets=127.15.129.133
--webserver_interface=127.15.129.133
--webserver_port=0
--tserver_master_addrs=127.15.129.190:40863
--builtin_ntp_servers=127.15.129.148:42717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20240502 07:58:07.589831 16502 heartbeater.cc:347] Connected to a master server at 127.15.129.190:40863
I20240502 07:58:07.590070 16502 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:07.590626 16502 heartbeater.cc:511] Master 127.15.129.190:40863 requested a full tablet report, sending...
I20240502 07:58:07.591645 15920 ts_manager.cc:194] Registered new tserver with Master: 48ab489bc1f54a65bd73c6c3cad9b246 (127.15.129.132:44329)
I20240502 07:58:07.592191 15920 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.132:43851
W20240502 07:58:07.698185 16505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:07.698408 16505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:07.698453 16505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:07.701529 16505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:07.701635 16505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.133
I20240502 07:58:07.705313 16505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:42717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.15.129.133
--webserver_port=0
--tserver_master_addrs=127.15.129.190:40863
--never_fsync=true
--heap_profile_path=/tmp/kudu.16505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.133
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:07.706282 16505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:07.707404 16505 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:07.716538 16510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:07.717839 16511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:07.718905 16513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:07.720088 16505 server_base.cc:1034] running on GCE node
I20240502 07:58:07.720481 16505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:07.721035 16505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:07.722193 16505 hybrid_clock.cc:648] HybridClock initialized: now 1714636687722170 us; error 44 us; skew 500 ppm
I20240502 07:58:07.724046 16505 webserver.cc:460] Webserver started at http://127.15.129.133:38055/ using document root <none> and password file <none>
I20240502 07:58:07.724601 16505 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:07.724660 16505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:07.724833 16505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:07.726505 16505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "8aff6a9d2d0b4697aeab141cc062d728"
format_stamp: "Formatted at 2024-05-02 07:58:07 on dist-test-slave-0hx2"
I20240502 07:58:07.726975 16505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "8aff6a9d2d0b4697aeab141cc062d728"
format_stamp: "Formatted at 2024-05-02 07:58:07 on dist-test-slave-0hx2"
I20240502 07:58:07.730268 16505 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:58:07.732553 16519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:07.733651 16505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:58:07.733767 16505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "8aff6a9d2d0b4697aeab141cc062d728"
format_stamp: "Formatted at 2024-05-02 07:58:07 on dist-test-slave-0hx2"
I20240502 07:58:07.733862 16505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:07.758422 16505 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:07.759114 16505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:07.759294 16505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:07.759816 16505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:07.760666 16505 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:07.760715 16505 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:07.760772 16505 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:07.760805 16505 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:07.769037 16505 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.133:38159
I20240502 07:58:07.769068 16632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.133:38159 every 8 connection(s)
I20240502 07:58:07.769984 16505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 07:58:07.772706 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 16505
I20240502 07:58:07.772835 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.AdminCliTest.RebalancerReportOnly.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 07:58:07.778743 16633 heartbeater.cc:347] Connected to a master server at 127.15.129.190:40863
I20240502 07:58:07.778952 16633 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:07.779426 16633 heartbeater.cc:511] Master 127.15.129.190:40863 requested a full tablet report, sending...
I20240502 07:58:07.780306 15920 ts_manager.cc:194] Registered new tserver with Master: 8aff6a9d2d0b4697aeab141cc062d728 (127.15.129.133:38159)
I20240502 07:58:07.780838 15920 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.133:58075
I20240502 07:58:07.787139 15878 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 07:58:07.798784 15920 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59044:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240502 07:58:07.816969 16567 tablet_service.cc:1467] Processing CreateTablet for tablet ac223f1df57445b4b4b853131bb9c6d9 (DEFAULT_TABLE table=TestTable [id=e210741bf830410d878313df2e308c04]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:07.816972 16436 tablet_service.cc:1467] Processing CreateTablet for tablet ac223f1df57445b4b4b853131bb9c6d9 (DEFAULT_TABLE table=TestTable [id=e210741bf830410d878313df2e308c04]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:07.816974 16043 tablet_service.cc:1467] Processing CreateTablet for tablet ac223f1df57445b4b4b853131bb9c6d9 (DEFAULT_TABLE table=TestTable [id=e210741bf830410d878313df2e308c04]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:07.817898 16436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac223f1df57445b4b4b853131bb9c6d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:07.817898 16567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac223f1df57445b4b4b853131bb9c6d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:07.817924 16043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac223f1df57445b4b4b853131bb9c6d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:07.823199 16650 tablet_bootstrap.cc:492] T ac223f1df57445b4b4b853131bb9c6d9 P 8aff6a9d2d0b4697aeab141cc062d728: Bootstrap starting.
I20240502 07:58:07.824115 16651 tablet_bootstrap.cc:492] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246: Bootstrap starting.
I20240502 07:58:07.824556 16652 tablet_bootstrap.cc:492] T ac223f1df57445b4b4b853131bb9c6d9 P f881831625a4439e88fbc46bb7582205: Bootstrap starting.
I20240502 07:58:07.826618 16650 tablet_bootstrap.cc:654] T ac223f1df57445b4b4b853131bb9c6d9 P 8aff6a9d2d0b4697aeab141cc062d728: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:07.826838 16651 tablet_bootstrap.cc:654] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:07.826874 16652 tablet_bootstrap.cc:654] T ac223f1df57445b4b4b853131bb9c6d9 P f881831625a4439e88fbc46bb7582205: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:07.827458 16651 log.cc:824] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:07.827481 16652 log.cc:824] T ac223f1df57445b4b4b853131bb9c6d9 P f881831625a4439e88fbc46bb7582205: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:07.827481 16650 log.cc:824] T ac223f1df57445b4b4b853131bb9c6d9 P 8aff6a9d2d0b4697aeab141cc062d728: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:07.829236 16652 tablet_bootstrap.cc:492] T ac223f1df57445b4b4b853131bb9c6d9 P f881831625a4439e88fbc46bb7582205: No bootstrap required, opened a new log
I20240502 07:58:07.829267 16650 tablet_bootstrap.cc:492] T ac223f1df57445b4b4b853131bb9c6d9 P 8aff6a9d2d0b4697aeab141cc062d728: No bootstrap required, opened a new log
I20240502 07:58:07.829267 16651 tablet_bootstrap.cc:492] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246: No bootstrap required, opened a new log
I20240502 07:58:07.829447 16652 ts_tablet_manager.cc:1367] T ac223f1df57445b4b4b853131bb9c6d9 P f881831625a4439e88fbc46bb7582205: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 07:58:07.829452 16651 ts_tablet_manager.cc:1367] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20240502 07:58:07.829582 16650 ts_tablet_manager.cc:1367] T ac223f1df57445b4b4b853131bb9c6d9 P 8aff6a9d2d0b4697aeab141cc062d728: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20240502 07:58:07.832351 16652 raft_consensus.cc:348] T ac223f1df57445b4b4b853131bb9c6d9 P f881831625a4439e88fbc46bb7582205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aff6a9d2d0b4697aeab141cc062d728" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 38159 } } peers { permanent_uuid: "f881831625a4439e88fbc46bb7582205" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 39041 } } peers { permanent_uuid: "48ab489bc1f54a65bd73c6c3cad9b246" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 44329 } }
I20240502 07:58:07.832548 16652 raft_consensus.cc:374] T ac223f1df57445b4b4b853131bb9c6d9 P f881831625a4439e88fbc46bb7582205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:07.832633 16652 raft_consensus.cc:729] T ac223f1df57445b4b4b853131bb9c6d9 P f881831625a4439e88fbc46bb7582205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f881831625a4439e88fbc46bb7582205, State: Initialized, Role: FOLLOWER
I20240502 07:58:07.833076 16652 consensus_queue.cc:260] T ac223f1df57445b4b4b853131bb9c6d9 P f881831625a4439e88fbc46bb7582205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aff6a9d2d0b4697aeab141cc062d728" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 38159 } } peers { permanent_uuid: "f881831625a4439e88fbc46bb7582205" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 39041 } } peers { permanent_uuid: "48ab489bc1f54a65bd73c6c3cad9b246" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 44329 } }
I20240502 07:58:07.833366 16650 raft_consensus.cc:348] T ac223f1df57445b4b4b853131bb9c6d9 P 8aff6a9d2d0b4697aeab141cc062d728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aff6a9d2d0b4697aeab141cc062d728" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 38159 } } peers { permanent_uuid: "f881831625a4439e88fbc46bb7582205" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 39041 } } peers { permanent_uuid: "48ab489bc1f54a65bd73c6c3cad9b246" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 44329 } }
I20240502 07:58:07.833366 16651 raft_consensus.cc:348] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aff6a9d2d0b4697aeab141cc062d728" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 38159 } } peers { permanent_uuid: "f881831625a4439e88fbc46bb7582205" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 39041 } } peers { permanent_uuid: "48ab489bc1f54a65bd73c6c3cad9b246" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 44329 } }
I20240502 07:58:07.833560 16651 raft_consensus.cc:374] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:07.833560 16650 raft_consensus.cc:374] T ac223f1df57445b4b4b853131bb9c6d9 P 8aff6a9d2d0b4697aeab141cc062d728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:07.833628 16650 raft_consensus.cc:729] T ac223f1df57445b4b4b853131bb9c6d9 P 8aff6a9d2d0b4697aeab141cc062d728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aff6a9d2d0b4697aeab141cc062d728, State: Initialized, Role: FOLLOWER
I20240502 07:58:07.833714 16651 raft_consensus.cc:729] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48ab489bc1f54a65bd73c6c3cad9b246, State: Initialized, Role: FOLLOWER
I20240502 07:58:07.833817 16109 heartbeater.cc:503] Master 127.15.129.190:40863 was elected leader, sending a full tablet report...
I20240502 07:58:07.834075 16650 consensus_queue.cc:260] T ac223f1df57445b4b4b853131bb9c6d9 P 8aff6a9d2d0b4697aeab141cc062d728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aff6a9d2d0b4697aeab141cc062d728" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 38159 } } peers { permanent_uuid: "f881831625a4439e88fbc46bb7582205" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 39041 } } peers { permanent_uuid: "48ab489bc1f54a65bd73c6c3cad9b246" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 44329 } }
I20240502 07:58:07.834123 16651 consensus_queue.cc:260] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aff6a9d2d0b4697aeab141cc062d728" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 38159 } } peers { permanent_uuid: "f881831625a4439e88fbc46bb7582205" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 39041 } } peers { permanent_uuid: "48ab489bc1f54a65bd73c6c3cad9b246" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 44329 } }
I20240502 07:58:07.834920 16650 ts_tablet_manager.cc:1398] T ac223f1df57445b4b4b853131bb9c6d9 P 8aff6a9d2d0b4697aeab141cc062d728: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 07:58:07.834910 16502 heartbeater.cc:503] Master 127.15.129.190:40863 was elected leader, sending a full tablet report...
I20240502 07:58:07.834939 16633 heartbeater.cc:503] Master 127.15.129.190:40863 was elected leader, sending a full tablet report...
I20240502 07:58:07.835305 16651 ts_tablet_manager.cc:1398] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240502 07:58:07.836735 16652 ts_tablet_manager.cc:1398] T ac223f1df57445b4b4b853131bb9c6d9 P f881831625a4439e88fbc46bb7582205: Time spent starting tablet: real 0.007s user 0.003s sys 0.001s
I20240502 07:58:07.956041 16657 raft_consensus.cc:482] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:07.956236 16657 raft_consensus.cc:504] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aff6a9d2d0b4697aeab141cc062d728" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 38159 } } peers { permanent_uuid: "f881831625a4439e88fbc46bb7582205" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 39041 } } peers { permanent_uuid: "48ab489bc1f54a65bd73c6c3cad9b246" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 44329 } }
I20240502 07:58:07.957443 16657 leader_election.cc:288] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8aff6a9d2d0b4697aeab141cc062d728 (127.15.129.133:38159), f881831625a4439e88fbc46bb7582205 (127.15.129.129:39041)
I20240502 07:58:07.961491 16063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac223f1df57445b4b4b853131bb9c6d9" candidate_uuid: "48ab489bc1f54a65bd73c6c3cad9b246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f881831625a4439e88fbc46bb7582205" is_pre_election: true
I20240502 07:58:07.961464 16587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac223f1df57445b4b4b853131bb9c6d9" candidate_uuid: "48ab489bc1f54a65bd73c6c3cad9b246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aff6a9d2d0b4697aeab141cc062d728" is_pre_election: true
I20240502 07:58:07.961792 16587 raft_consensus.cc:2454] T ac223f1df57445b4b4b853131bb9c6d9 P 8aff6a9d2d0b4697aeab141cc062d728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48ab489bc1f54a65bd73c6c3cad9b246 in term 0.
I20240502 07:58:07.961791 16063 raft_consensus.cc:2454] T ac223f1df57445b4b4b853131bb9c6d9 P f881831625a4439e88fbc46bb7582205 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48ab489bc1f54a65bd73c6c3cad9b246 in term 0.
I20240502 07:58:07.962247 16390 leader_election.cc:302] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48ab489bc1f54a65bd73c6c3cad9b246, f881831625a4439e88fbc46bb7582205; no voters:
I20240502 07:58:07.962481 16657 raft_consensus.cc:2781] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:07.962556 16657 raft_consensus.cc:482] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:07.962605 16657 raft_consensus.cc:3037] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246 [term 0 FOLLOWER]: Advancing to term 1
W20240502 07:58:08.003907 16110 tablet.cc:2369] T ac223f1df57445b4b4b853131bb9c6d9 P f881831625a4439e88fbc46bb7582205: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 07:58:08.020848 16634 tablet.cc:2369] T ac223f1df57445b4b4b853131bb9c6d9 P 8aff6a9d2d0b4697aeab141cc062d728: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:58:08.045019 16657 raft_consensus.cc:504] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aff6a9d2d0b4697aeab141cc062d728" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 38159 } } peers { permanent_uuid: "f881831625a4439e88fbc46bb7582205" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 39041 } } peers { permanent_uuid: "48ab489bc1f54a65bd73c6c3cad9b246" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 44329 } }
I20240502 07:58:08.045533 16657 leader_election.cc:288] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246 [CANDIDATE]: Term 1 election: Requested vote from peers 8aff6a9d2d0b4697aeab141cc062d728 (127.15.129.133:38159), f881831625a4439e88fbc46bb7582205 (127.15.129.129:39041)
I20240502 07:58:08.045976 16063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac223f1df57445b4b4b853131bb9c6d9" candidate_uuid: "48ab489bc1f54a65bd73c6c3cad9b246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f881831625a4439e88fbc46bb7582205"
I20240502 07:58:08.045998 16587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac223f1df57445b4b4b853131bb9c6d9" candidate_uuid: "48ab489bc1f54a65bd73c6c3cad9b246" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8aff6a9d2d0b4697aeab141cc062d728"
I20240502 07:58:08.046119 16063 raft_consensus.cc:3037] T ac223f1df57445b4b4b853131bb9c6d9 P f881831625a4439e88fbc46bb7582205 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:08.046173 16587 raft_consensus.cc:3037] T ac223f1df57445b4b4b853131bb9c6d9 P 8aff6a9d2d0b4697aeab141cc062d728 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:08.047098 16063 raft_consensus.cc:2454] T ac223f1df57445b4b4b853131bb9c6d9 P f881831625a4439e88fbc46bb7582205 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48ab489bc1f54a65bd73c6c3cad9b246 in term 1.
I20240502 07:58:08.047423 16390 leader_election.cc:302] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48ab489bc1f54a65bd73c6c3cad9b246, f881831625a4439e88fbc46bb7582205; no voters:
I20240502 07:58:08.047550 16587 raft_consensus.cc:2454] T ac223f1df57445b4b4b853131bb9c6d9 P 8aff6a9d2d0b4697aeab141cc062d728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48ab489bc1f54a65bd73c6c3cad9b246 in term 1.
I20240502 07:58:08.047607 16657 raft_consensus.cc:2781] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:08.047820 16657 raft_consensus.cc:686] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246 [term 1 LEADER]: Becoming Leader. State: Replica: 48ab489bc1f54a65bd73c6c3cad9b246, State: Running, Role: LEADER
I20240502 07:58:08.048182 16657 consensus_queue.cc:237] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aff6a9d2d0b4697aeab141cc062d728" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 38159 } } peers { permanent_uuid: "f881831625a4439e88fbc46bb7582205" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 39041 } } peers { permanent_uuid: "48ab489bc1f54a65bd73c6c3cad9b246" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 44329 } }
I20240502 07:58:08.051369 15919 catalog_manager.cc:5484] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48ab489bc1f54a65bd73c6c3cad9b246 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "48ab489bc1f54a65bd73c6c3cad9b246" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8aff6a9d2d0b4697aeab141cc062d728" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 38159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f881831625a4439e88fbc46bb7582205" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 39041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48ab489bc1f54a65bd73c6c3cad9b246" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 44329 } health_report { overall_health: HEALTHY } } }
W20240502 07:58:08.081097 16503 tablet.cc:2369] T ac223f1df57445b4b4b853131bb9c6d9 P 48ab489bc1f54a65bd73c6c3cad9b246: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:58:08.104336 15878 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 07:58:08.106310 15878 ts_itest-base.cc:233] Waiting for 1 tablets on tserver f881831625a4439e88fbc46bb7582205 to finish bootstrapping
I20240502 07:58:08.112371 15878 ts_itest-base.cc:233] Waiting for 0 tablets on tserver a9e5dfa96d1f4b7d89aea4ce3575439c to finish bootstrapping
I20240502 07:58:08.118932 15878 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 456c02424ee24e4987d07088be526ea4 to finish bootstrapping
I20240502 07:58:08.124517 15878 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 48ab489bc1f54a65bd73c6c3cad9b246 to finish bootstrapping
I20240502 07:58:08.129280 15878 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 8aff6a9d2d0b4697aeab141cc062d728 to finish bootstrapping
I20240502 07:58:08.192049 16240 heartbeater.cc:503] Master 127.15.129.190:40863 was elected leader, sending a full tablet report...
I20240502 07:58:08.330868 16305 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:08.331300 16436 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:08.331746 16174 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:08.334738 16567 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:08.337491 16043 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:08.401585 16371 heartbeater.cc:503] Master 127.15.129.190:40863 was elected leader, sending a full tablet report...
I20240502 07:58:08.429881 16305 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:08.430212 16436 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:08.430732 16567 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:08.431041 16043 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:08.431705 16174 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:08.476222 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 15981
I20240502 07:58:08.484529 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 16112
I20240502 07:58:08.490314 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 16243
I20240502 07:58:08.496531 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 16374
I20240502 07:58:08.502326 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 16505
I20240502 07:58:08.507933 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 15890
2024-05-02T07:58:08Z chronyd exiting
[ OK ] AdminCliTest.RebalancerReportOnly (1983 ms)
[----------] 1 test from AdminCliTest (1983 ms total)
[----------] 1 test from RebalanceStartCriteriaTest
[ RUN ] RebalanceStartCriteriaTest.TabletServerIsDown/0
I20240502 07:58:08.530862 15878 test_util.cc:268] Using random seed: -155378262
I20240502 07:58:08.531605 15878 ts_itest-base.cc:112] Starting cluster with:
I20240502 07:58:08.531646 15878 ts_itest-base.cc:113] --------------
I20240502 07:58:08.531667 15878 ts_itest-base.cc:114] 5 tablet servers
I20240502 07:58:08.531684 15878 ts_itest-base.cc:115] 3 replicas per TS
I20240502 07:58:08.531711 15878 ts_itest-base.cc:116] --------------
2024-05-02T07:58:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T07:58:08Z Disabled control of system clock
I20240502 07:58:08.539805 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.129.190:43009
--webserver_interface=127.15.129.190
--webserver_port=0
--builtin_ntp_servers=127.15.129.148:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.129.190:43009
--raft_prepare_replacement_before_eviction=false
--enable_range_replica_placement=false with env {}
W20240502 07:58:08.650920 16738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:08.651263 16738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:08.651377 16738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:08.655623 16738 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:08.655695 16738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 07:58:08.655715 16738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:08.655735 16738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 07:58:08.655752 16738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 07:58:08.659423 16738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.15.129.190:43009
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.129.190:43009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.15.129.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.16738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:08.660312 16738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:08.661523 16738 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:08.669713 16743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:08.671113 16746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:08.672102 16744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:08.673555 16738 server_base.cc:1034] running on GCE node
I20240502 07:58:08.674054 16738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:08.674989 16738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:08.676158 16738 hybrid_clock.cc:648] HybridClock initialized: now 1714636688676131 us; error 45 us; skew 500 ppm
I20240502 07:58:08.678647 16738 webserver.cc:460] Webserver started at http://127.15.129.190:36099/ using document root <none> and password file <none>
I20240502 07:58:08.679769 16738 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:08.679850 16738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:08.680059 16738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:08.681872 16738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "44fcd7c43c54497ca4d0235646b9ab85"
format_stamp: "Formatted at 2024-05-02 07:58:08 on dist-test-slave-0hx2"
I20240502 07:58:08.682338 16738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "44fcd7c43c54497ca4d0235646b9ab85"
format_stamp: "Formatted at 2024-05-02 07:58:08 on dist-test-slave-0hx2"
I20240502 07:58:08.685794 16738 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20240502 07:58:08.688019 16752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:08.689044 16738 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:08.689157 16738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
uuid: "44fcd7c43c54497ca4d0235646b9ab85"
format_stamp: "Formatted at 2024-05-02 07:58:08 on dist-test-slave-0hx2"
I20240502 07:58:08.689240 16738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:08.716636 16738 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:08.717312 16738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:08.717515 16738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:08.724364 16738 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.190:43009
I20240502 07:58:08.724385 16804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.190:43009 every 8 connection(s)
I20240502 07:58:08.725386 16738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 07:58:08.725628 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 16738
I20240502 07:58:08.725716 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 07:58:08.728572 16805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:08.733822 16805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85: Bootstrap starting.
I20240502 07:58:08.735785 16805 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:08.736447 16805 log.cc:824] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:08.738159 16805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85: No bootstrap required, opened a new log
I20240502 07:58:08.740767 16805 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44fcd7c43c54497ca4d0235646b9ab85" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 43009 } }
I20240502 07:58:08.740964 16805 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:08.741039 16805 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44fcd7c43c54497ca4d0235646b9ab85, State: Initialized, Role: FOLLOWER
I20240502 07:58:08.741492 16805 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44fcd7c43c54497ca4d0235646b9ab85" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 43009 } }
I20240502 07:58:08.741622 16805 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:08.741688 16805 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:08.741804 16805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:08.748257 16805 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44fcd7c43c54497ca4d0235646b9ab85" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 43009 } }
I20240502 07:58:08.748663 16805 leader_election.cc:302] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44fcd7c43c54497ca4d0235646b9ab85; no voters:
I20240502 07:58:08.749019 16805 leader_election.cc:288] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:08.749132 16810 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:08.749691 16810 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85 [term 1 LEADER]: Becoming Leader. State: Replica: 44fcd7c43c54497ca4d0235646b9ab85, State: Running, Role: LEADER
I20240502 07:58:08.750048 16810 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44fcd7c43c54497ca4d0235646b9ab85" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 43009 } }
I20240502 07:58:08.750201 16805 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 07:58:08.751348 16812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44fcd7c43c54497ca4d0235646b9ab85. Latest consensus state: current_term: 1 leader_uuid: "44fcd7c43c54497ca4d0235646b9ab85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44fcd7c43c54497ca4d0235646b9ab85" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 43009 } } }
I20240502 07:58:08.751458 16812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85 [sys.catalog]: This master's current role is: LEADER
I20240502 07:58:08.751619 16811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44fcd7c43c54497ca4d0235646b9ab85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44fcd7c43c54497ca4d0235646b9ab85" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 43009 } } }
I20240502 07:58:08.751706 16811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85 [sys.catalog]: This master's current role is: LEADER
I20240502 07:58:08.751823 16819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 07:58:08.754667 16819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 07:58:08.760008 16819 catalog_manager.cc:1347] Generated new cluster ID: 8e06cdf77184427f8d60d8bbd2d8d687
I20240502 07:58:08.760076 16819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 07:58:08.786378 16819 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 07:58:08.787333 16819 catalog_manager.cc:1504] Loading token signing keys...
I20240502 07:58:08.794895 16819 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 44fcd7c43c54497ca4d0235646b9ab85: Generated new TSK 0
I20240502 07:58:08.796041 16819 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 07:58:08.809170 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.129:0
--local_ip_for_outbound_sockets=127.15.129.129
--webserver_interface=127.15.129.129
--webserver_port=0
--tserver_master_addrs=127.15.129.190:43009
--builtin_ntp_servers=127.15.129.148:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 07:58:08.926595 16829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:08.926820 16829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:08.926863 16829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:08.929891 16829 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:08.929955 16829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:08.930030 16829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.129
I20240502 07:58:08.933743 16829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.15.129.129
--webserver_port=0
--tserver_master_addrs=127.15.129.190:43009
--never_fsync=true
--heap_profile_path=/tmp/kudu.16829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.129
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:08.934660 16829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:08.935752 16829 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:08.945652 16835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:08.946521 16834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:08.947882 16837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:08.949101 16829 server_base.cc:1034] running on GCE node
I20240502 07:58:08.949615 16829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:08.950379 16829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:08.951617 16829 hybrid_clock.cc:648] HybridClock initialized: now 1714636688951600 us; error 37 us; skew 500 ppm
I20240502 07:58:08.953814 16829 webserver.cc:460] Webserver started at http://127.15.129.129:38055/ using document root <none> and password file <none>
I20240502 07:58:08.954391 16829 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:08.954457 16829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:08.954641 16829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:08.956374 16829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "e4a27c860e514bfea29fff8f19177b4b"
format_stamp: "Formatted at 2024-05-02 07:58:08 on dist-test-slave-0hx2"
I20240502 07:58:08.956861 16829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "e4a27c860e514bfea29fff8f19177b4b"
format_stamp: "Formatted at 2024-05-02 07:58:08 on dist-test-slave-0hx2"
I20240502 07:58:08.960309 16829 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20240502 07:58:08.962618 16843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:08.963660 16829 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:08.963771 16829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "e4a27c860e514bfea29fff8f19177b4b"
format_stamp: "Formatted at 2024-05-02 07:58:08 on dist-test-slave-0hx2"
I20240502 07:58:08.963865 16829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:08.986596 16829 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:08.987244 16829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:08.987408 16829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:08.987972 16829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:08.988893 16829 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:08.988940 16829 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:08.988999 16829 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:08.989027 16829 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:08.997632 16829 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.129:38383
I20240502 07:58:08.997660 16956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.129:38383 every 8 connection(s)
I20240502 07:58:08.998548 16829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 07:58:09.005970 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 16829
I20240502 07:58:09.006091 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 07:58:09.007064 16957 heartbeater.cc:347] Connected to a master server at 127.15.129.190:43009
I20240502 07:58:09.007275 16957 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:09.007779 16957 heartbeater.cc:511] Master 127.15.129.190:43009 requested a full tablet report, sending...
I20240502 07:58:09.008227 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.130:0
--local_ip_for_outbound_sockets=127.15.129.130
--webserver_interface=127.15.129.130
--webserver_port=0
--tserver_master_addrs=127.15.129.190:43009
--builtin_ntp_servers=127.15.129.148:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:09.009148 16767 ts_manager.cc:194] Registered new tserver with Master: e4a27c860e514bfea29fff8f19177b4b (127.15.129.129:38383)
I20240502 07:58:09.010576 16767 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.129:40165
W20240502 07:58:09.113834 16960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:09.114063 16960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:09.114140 16960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:09.117087 16960 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:09.117156 16960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:09.117261 16960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.130
I20240502 07:58:09.120784 16960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.15.129.130
--webserver_port=0
--tserver_master_addrs=127.15.129.190:43009
--never_fsync=true
--heap_profile_path=/tmp/kudu.16960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.130
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:09.121685 16960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:09.122817 16960 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:09.132236 16965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:09.133126 16966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:09.134567 16968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:09.135622 16960 server_base.cc:1034] running on GCE node
I20240502 07:58:09.136075 16960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:09.136703 16960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:09.137864 16960 hybrid_clock.cc:648] HybridClock initialized: now 1714636689137860 us; error 36 us; skew 500 ppm
I20240502 07:58:09.139897 16960 webserver.cc:460] Webserver started at http://127.15.129.130:37259/ using document root <none> and password file <none>
I20240502 07:58:09.140501 16960 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:09.140568 16960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:09.140759 16960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:09.142663 16960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "62a0e92fb61243fbbb346aafd8aeaa50"
format_stamp: "Formatted at 2024-05-02 07:58:09 on dist-test-slave-0hx2"
I20240502 07:58:09.143157 16960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "62a0e92fb61243fbbb346aafd8aeaa50"
format_stamp: "Formatted at 2024-05-02 07:58:09 on dist-test-slave-0hx2"
I20240502 07:58:09.146857 16960 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:58:09.149617 16974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:09.150810 16960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20240502 07:58:09.150946 16960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "62a0e92fb61243fbbb346aafd8aeaa50"
format_stamp: "Formatted at 2024-05-02 07:58:09 on dist-test-slave-0hx2"
I20240502 07:58:09.151074 16960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:09.168586 16960 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:09.169198 16960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:09.169396 16960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:09.169934 16960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:09.170814 16960 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:09.170864 16960 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:09.170925 16960 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:09.170953 16960 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:09.179145 16960 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.130:40495
I20240502 07:58:09.179170 17087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.130:40495 every 8 connection(s)
I20240502 07:58:09.180061 16960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 07:58:09.184998 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 16960
I20240502 07:58:09.185104 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 07:58:09.187001 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.131:0
--local_ip_for_outbound_sockets=127.15.129.131
--webserver_interface=127.15.129.131
--webserver_port=0
--tserver_master_addrs=127.15.129.190:43009
--builtin_ntp_servers=127.15.129.148:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:09.188714 17088 heartbeater.cc:347] Connected to a master server at 127.15.129.190:43009
I20240502 07:58:09.188947 17088 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:09.189374 17088 heartbeater.cc:511] Master 127.15.129.190:43009 requested a full tablet report, sending...
I20240502 07:58:09.190218 16767 ts_manager.cc:194] Registered new tserver with Master: 62a0e92fb61243fbbb346aafd8aeaa50 (127.15.129.130:40495)
I20240502 07:58:09.190766 16767 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.130:51907
W20240502 07:58:09.293517 17091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:09.293762 17091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:09.293807 17091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:09.297025 17091 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:09.297096 17091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:09.297192 17091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.131
I20240502 07:58:09.300720 17091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.15.129.131
--webserver_port=0
--tserver_master_addrs=127.15.129.190:43009
--never_fsync=true
--heap_profile_path=/tmp/kudu.17091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.131
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:09.301609 17091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:09.302701 17091 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:09.311830 17096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:09.313117 17099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:09.314199 17091 server_base.cc:1034] running on GCE node
W20240502 07:58:09.315353 17097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:09.316227 17091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:09.316913 17091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:09.318079 17091 hybrid_clock.cc:648] HybridClock initialized: now 1714636689318055 us; error 46 us; skew 500 ppm
I20240502 07:58:09.319895 17091 webserver.cc:460] Webserver started at http://127.15.129.131:44659/ using document root <none> and password file <none>
I20240502 07:58:09.320421 17091 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:09.320482 17091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:09.320652 17091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:09.322299 17091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "35830cc28e294f1a822a723969be999e"
format_stamp: "Formatted at 2024-05-02 07:58:09 on dist-test-slave-0hx2"
I20240502 07:58:09.322777 17091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "35830cc28e294f1a822a723969be999e"
format_stamp: "Formatted at 2024-05-02 07:58:09 on dist-test-slave-0hx2"
I20240502 07:58:09.326045 17091 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:58:09.328153 17105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:09.329126 17091 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:09.329216 17091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "35830cc28e294f1a822a723969be999e"
format_stamp: "Formatted at 2024-05-02 07:58:09 on dist-test-slave-0hx2"
I20240502 07:58:09.329349 17091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:09.342998 17091 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:09.343530 17091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:09.343669 17091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:09.344146 17091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:09.344949 17091 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:09.344992 17091 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:09.345027 17091 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:09.345042 17091 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:09.353555 17091 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.131:33263
I20240502 07:58:09.353580 17218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.131:33263 every 8 connection(s)
I20240502 07:58:09.354480 17091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 07:58:09.363210 17219 heartbeater.cc:347] Connected to a master server at 127.15.129.190:43009
I20240502 07:58:09.363446 17219 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:09.363489 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 17091
I20240502 07:58:09.363586 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 07:58:09.363919 17219 heartbeater.cc:511] Master 127.15.129.190:43009 requested a full tablet report, sending...
I20240502 07:58:09.364940 16769 ts_manager.cc:194] Registered new tserver with Master: 35830cc28e294f1a822a723969be999e (127.15.129.131:33263)
I20240502 07:58:09.365551 16769 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.131:42433
I20240502 07:58:09.365563 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.132:0
--local_ip_for_outbound_sockets=127.15.129.132
--webserver_interface=127.15.129.132
--webserver_port=0
--tserver_master_addrs=127.15.129.190:43009
--builtin_ntp_servers=127.15.129.148:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 07:58:09.471866 17222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:09.472080 17222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:09.472131 17222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:09.475149 17222 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:09.475208 17222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:09.475301 17222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.132
I20240502 07:58:09.478812 17222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.15.129.132
--webserver_port=0
--tserver_master_addrs=127.15.129.190:43009
--never_fsync=true
--heap_profile_path=/tmp/kudu.17222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.132
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:09.479645 17222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:09.480734 17222 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:09.489868 17230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:09.490792 17228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:09.492192 17227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:09.493474 17222 server_base.cc:1034] running on GCE node
I20240502 07:58:09.493862 17222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:09.494414 17222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:09.495579 17222 hybrid_clock.cc:648] HybridClock initialized: now 1714636689495578 us; error 36 us; skew 500 ppm
I20240502 07:58:09.497565 17222 webserver.cc:460] Webserver started at http://127.15.129.132:40267/ using document root <none> and password file <none>
I20240502 07:58:09.498096 17222 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:09.498157 17222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:09.498332 17222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:09.499975 17222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "a6b160716ced4beea454638110149e9e"
format_stamp: "Formatted at 2024-05-02 07:58:09 on dist-test-slave-0hx2"
I20240502 07:58:09.500416 17222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "a6b160716ced4beea454638110149e9e"
format_stamp: "Formatted at 2024-05-02 07:58:09 on dist-test-slave-0hx2"
I20240502 07:58:09.503666 17222 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20240502 07:58:09.505802 17236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:09.506805 17222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:09.506904 17222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "a6b160716ced4beea454638110149e9e"
format_stamp: "Formatted at 2024-05-02 07:58:09 on dist-test-slave-0hx2"
I20240502 07:58:09.506986 17222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:09.524768 17222 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:09.525404 17222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:09.525560 17222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:09.526067 17222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:09.526937 17222 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:09.526984 17222 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:09.527038 17222 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:09.527069 17222 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:09.535804 17222 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.132:41719
I20240502 07:58:09.535832 17349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.132:41719 every 8 connection(s)
I20240502 07:58:09.536705 17222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 07:58:09.541430 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 17222
I20240502 07:58:09.541515 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 07:58:09.543710 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.133:0
--local_ip_for_outbound_sockets=127.15.129.133
--webserver_interface=127.15.129.133
--webserver_port=0
--tserver_master_addrs=127.15.129.190:43009
--builtin_ntp_servers=127.15.129.148:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:09.545432 17350 heartbeater.cc:347] Connected to a master server at 127.15.129.190:43009
I20240502 07:58:09.545640 17350 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:09.546053 17350 heartbeater.cc:511] Master 127.15.129.190:43009 requested a full tablet report, sending...
I20240502 07:58:09.546963 16769 ts_manager.cc:194] Registered new tserver with Master: a6b160716ced4beea454638110149e9e (127.15.129.132:41719)
I20240502 07:58:09.547479 16769 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.132:39831
W20240502 07:58:09.651497 17353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:09.651731 17353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:09.651799 17353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:09.654999 17353 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:09.655081 17353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:09.655172 17353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.133
I20240502 07:58:09.659644 17353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.15.129.133
--webserver_port=0
--tserver_master_addrs=127.15.129.190:43009
--never_fsync=true
--heap_profile_path=/tmp/kudu.17353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.133
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:09.660624 17353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:09.661772 17353 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:09.671152 17361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:09.671977 17358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:09.673307 17359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:09.674482 17353 server_base.cc:1034] running on GCE node
I20240502 07:58:09.674899 17353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:09.675491 17353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:09.676666 17353 hybrid_clock.cc:648] HybridClock initialized: now 1714636689676634 us; error 46 us; skew 500 ppm
I20240502 07:58:09.678969 17353 webserver.cc:460] Webserver started at http://127.15.129.133:34143/ using document root <none> and password file <none>
I20240502 07:58:09.679603 17353 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:09.679661 17353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:09.679814 17353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:09.681514 17353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "6331497cbd144657887deca1c122eb97"
format_stamp: "Formatted at 2024-05-02 07:58:09 on dist-test-slave-0hx2"
I20240502 07:58:09.681931 17353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "6331497cbd144657887deca1c122eb97"
format_stamp: "Formatted at 2024-05-02 07:58:09 on dist-test-slave-0hx2"
I20240502 07:58:09.685148 17353 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 07:58:09.687359 17367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:09.688376 17353 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:09.688469 17353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "6331497cbd144657887deca1c122eb97"
format_stamp: "Formatted at 2024-05-02 07:58:09 on dist-test-slave-0hx2"
I20240502 07:58:09.688544 17353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:09.708782 17353 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:09.709403 17353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:09.709538 17353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:09.710105 17353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:09.711009 17353 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:09.711071 17353 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:09.711127 17353 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:09.711181 17353 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:09.719501 17353 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.133:41245
I20240502 07:58:09.719522 17480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.133:41245 every 8 connection(s)
I20240502 07:58:09.720470 17353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 07:58:09.729147 17481 heartbeater.cc:347] Connected to a master server at 127.15.129.190:43009
I20240502 07:58:09.729398 17481 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:09.729866 17481 heartbeater.cc:511] Master 127.15.129.190:43009 requested a full tablet report, sending...
I20240502 07:58:09.730062 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 17353
I20240502 07:58:09.730157 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceStartCriteriaTest.TabletServerIsDown_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 07:58:09.730868 16769 ts_manager.cc:194] Registered new tserver with Master: 6331497cbd144657887deca1c122eb97 (127.15.129.133:41245)
I20240502 07:58:09.731534 16769 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.133:49867
I20240502 07:58:09.732409 15878 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 07:58:09.740777 16767 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34054:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240502 07:58:09.758344 17022 tablet_service.cc:1467] Processing CreateTablet for tablet b955df1e8f954b3398d76edca8cf78c1 (DEFAULT_TABLE table=TestTable [id=a7a0b75cca4b4d9f8789e6a729753729]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:09.758333 17153 tablet_service.cc:1467] Processing CreateTablet for tablet b955df1e8f954b3398d76edca8cf78c1 (DEFAULT_TABLE table=TestTable [id=a7a0b75cca4b4d9f8789e6a729753729]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:09.758333 17284 tablet_service.cc:1467] Processing CreateTablet for tablet b955df1e8f954b3398d76edca8cf78c1 (DEFAULT_TABLE table=TestTable [id=a7a0b75cca4b4d9f8789e6a729753729]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:09.759297 17022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b955df1e8f954b3398d76edca8cf78c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:09.759497 17153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b955df1e8f954b3398d76edca8cf78c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:09.759497 17284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b955df1e8f954b3398d76edca8cf78c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:09.764446 17498 tablet_bootstrap.cc:492] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50: Bootstrap starting.
I20240502 07:58:09.765758 17499 tablet_bootstrap.cc:492] T b955df1e8f954b3398d76edca8cf78c1 P a6b160716ced4beea454638110149e9e: Bootstrap starting.
I20240502 07:58:09.767001 17498 tablet_bootstrap.cc:654] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:09.767325 17499 tablet_bootstrap.cc:654] T b955df1e8f954b3398d76edca8cf78c1 P a6b160716ced4beea454638110149e9e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:09.767683 17500 tablet_bootstrap.cc:492] T b955df1e8f954b3398d76edca8cf78c1 P 35830cc28e294f1a822a723969be999e: Bootstrap starting.
I20240502 07:58:09.767719 17498 log.cc:824] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:09.769186 17498 tablet_bootstrap.cc:492] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50: No bootstrap required, opened a new log
I20240502 07:58:09.769253 17500 tablet_bootstrap.cc:654] T b955df1e8f954b3398d76edca8cf78c1 P 35830cc28e294f1a822a723969be999e: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:09.769358 17498 ts_tablet_manager.cc:1367] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 07:58:09.769922 17500 log.cc:824] T b955df1e8f954b3398d76edca8cf78c1 P 35830cc28e294f1a822a723969be999e: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:09.770987 17499 log.cc:824] T b955df1e8f954b3398d76edca8cf78c1 P a6b160716ced4beea454638110149e9e: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:09.772001 17500 tablet_bootstrap.cc:492] T b955df1e8f954b3398d76edca8cf78c1 P 35830cc28e294f1a822a723969be999e: No bootstrap required, opened a new log
I20240502 07:58:09.772213 17500 ts_tablet_manager.cc:1367] T b955df1e8f954b3398d76edca8cf78c1 P 35830cc28e294f1a822a723969be999e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 07:58:09.772403 17499 tablet_bootstrap.cc:492] T b955df1e8f954b3398d76edca8cf78c1 P a6b160716ced4beea454638110149e9e: No bootstrap required, opened a new log
I20240502 07:58:09.772562 17499 ts_tablet_manager.cc:1367] T b955df1e8f954b3398d76edca8cf78c1 P a6b160716ced4beea454638110149e9e: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20240502 07:58:09.772954 17498 raft_consensus.cc:348] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0e92fb61243fbbb346aafd8aeaa50" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 40495 } } peers { permanent_uuid: "a6b160716ced4beea454638110149e9e" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41719 } } peers { permanent_uuid: "35830cc28e294f1a822a723969be999e" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 33263 } }
I20240502 07:58:09.773171 17498 raft_consensus.cc:374] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:09.773226 17498 raft_consensus.cc:729] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a0e92fb61243fbbb346aafd8aeaa50, State: Initialized, Role: FOLLOWER
I20240502 07:58:09.773725 17498 consensus_queue.cc:260] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0e92fb61243fbbb346aafd8aeaa50" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 40495 } } peers { permanent_uuid: "a6b160716ced4beea454638110149e9e" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41719 } } peers { permanent_uuid: "35830cc28e294f1a822a723969be999e" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 33263 } }
I20240502 07:58:09.774510 17088 heartbeater.cc:503] Master 127.15.129.190:43009 was elected leader, sending a full tablet report...
I20240502 07:58:09.775768 17498 ts_tablet_manager.cc:1398] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20240502 07:58:09.775671 17499 raft_consensus.cc:348] T b955df1e8f954b3398d76edca8cf78c1 P a6b160716ced4beea454638110149e9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0e92fb61243fbbb346aafd8aeaa50" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 40495 } } peers { permanent_uuid: "a6b160716ced4beea454638110149e9e" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41719 } } peers { permanent_uuid: "35830cc28e294f1a822a723969be999e" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 33263 } }
I20240502 07:58:09.775862 17499 raft_consensus.cc:374] T b955df1e8f954b3398d76edca8cf78c1 P a6b160716ced4beea454638110149e9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:09.775920 17499 raft_consensus.cc:729] T b955df1e8f954b3398d76edca8cf78c1 P a6b160716ced4beea454638110149e9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6b160716ced4beea454638110149e9e, State: Initialized, Role: FOLLOWER
I20240502 07:58:09.776322 17499 consensus_queue.cc:260] T b955df1e8f954b3398d76edca8cf78c1 P a6b160716ced4beea454638110149e9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0e92fb61243fbbb346aafd8aeaa50" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 40495 } } peers { permanent_uuid: "a6b160716ced4beea454638110149e9e" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41719 } } peers { permanent_uuid: "35830cc28e294f1a822a723969be999e" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 33263 } }
I20240502 07:58:09.776594 17500 raft_consensus.cc:348] T b955df1e8f954b3398d76edca8cf78c1 P 35830cc28e294f1a822a723969be999e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0e92fb61243fbbb346aafd8aeaa50" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 40495 } } peers { permanent_uuid: "a6b160716ced4beea454638110149e9e" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41719 } } peers { permanent_uuid: "35830cc28e294f1a822a723969be999e" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 33263 } }
I20240502 07:58:09.776795 17500 raft_consensus.cc:374] T b955df1e8f954b3398d76edca8cf78c1 P 35830cc28e294f1a822a723969be999e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:09.776859 17500 raft_consensus.cc:729] T b955df1e8f954b3398d76edca8cf78c1 P 35830cc28e294f1a822a723969be999e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35830cc28e294f1a822a723969be999e, State: Initialized, Role: FOLLOWER
I20240502 07:58:09.777230 17350 heartbeater.cc:503] Master 127.15.129.190:43009 was elected leader, sending a full tablet report...
I20240502 07:58:09.777245 17499 ts_tablet_manager.cc:1398] T b955df1e8f954b3398d76edca8cf78c1 P a6b160716ced4beea454638110149e9e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 07:58:09.777251 17500 consensus_queue.cc:260] T b955df1e8f954b3398d76edca8cf78c1 P 35830cc28e294f1a822a723969be999e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0e92fb61243fbbb346aafd8aeaa50" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 40495 } } peers { permanent_uuid: "a6b160716ced4beea454638110149e9e" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41719 } } peers { permanent_uuid: "35830cc28e294f1a822a723969be999e" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 33263 } }
I20240502 07:58:09.778041 17500 ts_tablet_manager.cc:1398] T b955df1e8f954b3398d76edca8cf78c1 P 35830cc28e294f1a822a723969be999e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240502 07:58:09.778381 17219 heartbeater.cc:503] Master 127.15.129.190:43009 was elected leader, sending a full tablet report...
I20240502 07:58:09.785920 17504 raft_consensus.cc:482] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:09.786118 17504 raft_consensus.cc:504] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0e92fb61243fbbb346aafd8aeaa50" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 40495 } } peers { permanent_uuid: "a6b160716ced4beea454638110149e9e" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41719 } } peers { permanent_uuid: "35830cc28e294f1a822a723969be999e" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 33263 } }
I20240502 07:58:09.787118 17504 leader_election.cc:288] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6b160716ced4beea454638110149e9e (127.15.129.132:41719), 35830cc28e294f1a822a723969be999e (127.15.129.131:33263)
W20240502 07:58:09.787618 17351 tablet.cc:2369] T b955df1e8f954b3398d76edca8cf78c1 P a6b160716ced4beea454638110149e9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:58:09.790874 17173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b955df1e8f954b3398d76edca8cf78c1" candidate_uuid: "62a0e92fb61243fbbb346aafd8aeaa50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35830cc28e294f1a822a723969be999e" is_pre_election: true
I20240502 07:58:09.790906 17304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b955df1e8f954b3398d76edca8cf78c1" candidate_uuid: "62a0e92fb61243fbbb346aafd8aeaa50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6b160716ced4beea454638110149e9e" is_pre_election: true
I20240502 07:58:09.791154 17304 raft_consensus.cc:2454] T b955df1e8f954b3398d76edca8cf78c1 P a6b160716ced4beea454638110149e9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a0e92fb61243fbbb346aafd8aeaa50 in term 0.
I20240502 07:58:09.791148 17173 raft_consensus.cc:2454] T b955df1e8f954b3398d76edca8cf78c1 P 35830cc28e294f1a822a723969be999e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62a0e92fb61243fbbb346aafd8aeaa50 in term 0.
I20240502 07:58:09.791602 16976 leader_election.cc:302] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a0e92fb61243fbbb346aafd8aeaa50, a6b160716ced4beea454638110149e9e; no voters:
I20240502 07:58:09.791949 17504 raft_consensus.cc:2781] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:09.792014 17504 raft_consensus.cc:482] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:09.792042 17504 raft_consensus.cc:3037] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:09.792932 17504 raft_consensus.cc:504] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0e92fb61243fbbb346aafd8aeaa50" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 40495 } } peers { permanent_uuid: "a6b160716ced4beea454638110149e9e" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41719 } } peers { permanent_uuid: "35830cc28e294f1a822a723969be999e" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 33263 } }
I20240502 07:58:09.793256 17504 leader_election.cc:288] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [CANDIDATE]: Term 1 election: Requested vote from peers a6b160716ced4beea454638110149e9e (127.15.129.132:41719), 35830cc28e294f1a822a723969be999e (127.15.129.131:33263)
I20240502 07:58:09.793641 17304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b955df1e8f954b3398d76edca8cf78c1" candidate_uuid: "62a0e92fb61243fbbb346aafd8aeaa50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6b160716ced4beea454638110149e9e"
I20240502 07:58:09.793641 17173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b955df1e8f954b3398d76edca8cf78c1" candidate_uuid: "62a0e92fb61243fbbb346aafd8aeaa50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35830cc28e294f1a822a723969be999e"
I20240502 07:58:09.793761 17304 raft_consensus.cc:3037] T b955df1e8f954b3398d76edca8cf78c1 P a6b160716ced4beea454638110149e9e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:09.793762 17173 raft_consensus.cc:3037] T b955df1e8f954b3398d76edca8cf78c1 P 35830cc28e294f1a822a723969be999e [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:09.794782 17304 raft_consensus.cc:2454] T b955df1e8f954b3398d76edca8cf78c1 P a6b160716ced4beea454638110149e9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a0e92fb61243fbbb346aafd8aeaa50 in term 1.
I20240502 07:58:09.794782 17173 raft_consensus.cc:2454] T b955df1e8f954b3398d76edca8cf78c1 P 35830cc28e294f1a822a723969be999e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62a0e92fb61243fbbb346aafd8aeaa50 in term 1.
I20240502 07:58:09.795102 16978 leader_election.cc:302] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35830cc28e294f1a822a723969be999e, 62a0e92fb61243fbbb346aafd8aeaa50; no voters:
I20240502 07:58:09.795325 17504 raft_consensus.cc:2781] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:09.795548 17504 raft_consensus.cc:686] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [term 1 LEADER]: Becoming Leader. State: Replica: 62a0e92fb61243fbbb346aafd8aeaa50, State: Running, Role: LEADER
I20240502 07:58:09.795886 17504 consensus_queue.cc:237] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0e92fb61243fbbb346aafd8aeaa50" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 40495 } } peers { permanent_uuid: "a6b160716ced4beea454638110149e9e" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41719 } } peers { permanent_uuid: "35830cc28e294f1a822a723969be999e" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 33263 } }
I20240502 07:58:09.798527 16767 catalog_manager.cc:5484] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62a0e92fb61243fbbb346aafd8aeaa50 (127.15.129.130). New cstate: current_term: 1 leader_uuid: "62a0e92fb61243fbbb346aafd8aeaa50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a0e92fb61243fbbb346aafd8aeaa50" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 40495 } } peers { permanent_uuid: "a6b160716ced4beea454638110149e9e" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41719 } } peers { permanent_uuid: "35830cc28e294f1a822a723969be999e" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 33263 } } }
I20240502 07:58:09.805069 15878 external_mini_cluster.cc:930] 5 TS(s) registered with all masters
I20240502 07:58:09.806336 15878 ts_itest-base.cc:233] Waiting for 0 tablets on tserver e4a27c860e514bfea29fff8f19177b4b to finish bootstrapping
I20240502 07:58:09.812093 15878 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 62a0e92fb61243fbbb346aafd8aeaa50 to finish bootstrapping
I20240502 07:58:09.816639 15878 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 35830cc28e294f1a822a723969be999e to finish bootstrapping
I20240502 07:58:09.820930 15878 ts_itest-base.cc:233] Waiting for 1 tablets on tserver a6b160716ced4beea454638110149e9e to finish bootstrapping
I20240502 07:58:09.825407 15878 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 6331497cbd144657887deca1c122eb97 to finish bootstrapping
I20240502 07:58:09.830952 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 16829
W20240502 07:58:09.855536 17220 tablet.cc:2369] T b955df1e8f954b3398d76edca8cf78c1 P 35830cc28e294f1a822a723969be999e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 07:58:09.931275 17089 tablet.cc:2369] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:58:10.021160 17153 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:10.021374 17284 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:10.023427 17415 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:10.023449 17022 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:10.098788 17415 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:10.098825 17153 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:10.099574 17022 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:10.100103 17284 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:10.187089 17504 consensus_queue.cc:1035] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35830cc28e294f1a822a723969be999e" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:10.192595 17509 consensus_queue.cc:1035] T b955df1e8f954b3398d76edca8cf78c1 P 62a0e92fb61243fbbb346aafd8aeaa50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6b160716ced4beea454638110149e9e" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:10.199514 17153 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:10.203955 17284 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:10.205483 17022 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:10.211439 17415 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:10.426596 17153 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:10.427044 17022 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:10.427037 17284 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:10.427158 17415 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:10.503086 17153 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:10.504341 17022 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:10.504456 17284 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:10.504890 17415 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:10.615522 17415 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:10.615953 17153 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:10.616290 17284 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:10.616456 17022 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:10.648600 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 16960
I20240502 07:58:10.654824 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 17091
I20240502 07:58:10.660264 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 17222
I20240502 07:58:10.666054 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 17353
I20240502 07:58:10.671255 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 16738
2024-05-02T07:58:10Z chronyd exiting
[ OK ] RebalanceStartCriteriaTest.TabletServerIsDown/0 (2163 ms)
[----------] 1 test from RebalanceStartCriteriaTest (2163 ms total)
[----------] 1 test from RebalanceParamTest
[ RUN ] RebalanceParamTest.Rebalance/0
I20240502 07:58:10.694201 15878 test_util.cc:268] Using random seed: -153214926
I20240502 07:58:10.695111 15878 ts_itest-base.cc:112] Starting cluster with:
I20240502 07:58:10.695160 15878 ts_itest-base.cc:113] --------------
I20240502 07:58:10.695175 15878 ts_itest-base.cc:114] 7 tablet servers
I20240502 07:58:10.695189 15878 ts_itest-base.cc:115] 1 replicas per TS
I20240502 07:58:10.695214 15878 ts_itest-base.cc:116] --------------
2024-05-02T07:58:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T07:58:10Z Disabled control of system clock
I20240502 07:58:10.705797 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.129.190:41083
--webserver_interface=127.15.129.190
--webserver_port=0
--builtin_ntp_servers=127.15.129.148:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.129.190:41083
--allow_unsafe_replication_factor
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=false
--tserver_unresponsive_timeout_ms=3000 with env {}
W20240502 07:58:10.813637 17701 flags.cc:425] Enabled unsafe flag: --allow_unsafe_replication_factor=true
W20240502 07:58:10.813894 17701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:10.813921 17701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:10.813959 17701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:10.817142 17701 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:10.817247 17701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 07:58:10.817307 17701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:10.817377 17701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 07:58:10.817415 17701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 07:58:10.821475 17701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
--allow_unsafe_replication_factor=true
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.15.129.190:41083
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.129.190:41083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.15.129.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.17701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:10.822432 17701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:10.823565 17701 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:10.832077 17709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:10.832963 17706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:10.834302 17707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:10.835623 17701 server_base.cc:1034] running on GCE node
I20240502 07:58:10.836136 17701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:10.837074 17701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:10.838253 17701 hybrid_clock.cc:648] HybridClock initialized: now 1714636690838215 us; error 52 us; skew 500 ppm
I20240502 07:58:10.840281 17701 webserver.cc:460] Webserver started at http://127.15.129.190:37961/ using document root <none> and password file <none>
I20240502 07:58:10.840865 17701 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:10.840919 17701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:10.841074 17701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:10.842756 17701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "53be75cf33b74f38884080e1936c65f1"
format_stamp: "Formatted at 2024-05-02 07:58:10 on dist-test-slave-0hx2"
I20240502 07:58:10.843217 17701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "53be75cf33b74f38884080e1936c65f1"
format_stamp: "Formatted at 2024-05-02 07:58:10 on dist-test-slave-0hx2"
I20240502 07:58:10.846549 17701 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:58:10.848695 17715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:10.849752 17701 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:58:10.849867 17701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
uuid: "53be75cf33b74f38884080e1936c65f1"
format_stamp: "Formatted at 2024-05-02 07:58:10 on dist-test-slave-0hx2"
I20240502 07:58:10.849953 17701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:10.880118 17701 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:10.880781 17701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:10.880944 17701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:10.888127 17701 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.190:41083
I20240502 07:58:10.888134 17767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.190:41083 every 8 connection(s)
I20240502 07:58:10.889142 17701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 07:58:10.891332 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 17701
I20240502 07:58:10.891430 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 07:58:10.891822 17768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:10.897876 17768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1: Bootstrap starting.
I20240502 07:58:10.899900 17768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:10.900586 17768 log.cc:824] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:10.902204 17768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1: No bootstrap required, opened a new log
I20240502 07:58:10.904582 17768 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53be75cf33b74f38884080e1936c65f1" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 41083 } }
I20240502 07:58:10.904765 17768 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:10.904827 17768 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53be75cf33b74f38884080e1936c65f1, State: Initialized, Role: FOLLOWER
I20240502 07:58:10.905200 17768 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53be75cf33b74f38884080e1936c65f1" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 41083 } }
I20240502 07:58:10.905311 17768 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:10.905396 17768 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:10.905501 17768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:10.906318 17768 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53be75cf33b74f38884080e1936c65f1" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 41083 } }
I20240502 07:58:10.906628 17768 leader_election.cc:302] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53be75cf33b74f38884080e1936c65f1; no voters:
I20240502 07:58:10.906879 17768 leader_election.cc:288] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:10.907316 17773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:10.907544 17773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1 [term 1 LEADER]: Becoming Leader. State: Replica: 53be75cf33b74f38884080e1936c65f1, State: Running, Role: LEADER
I20240502 07:58:10.907829 17773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53be75cf33b74f38884080e1936c65f1" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 41083 } }
I20240502 07:58:10.907917 17768 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 07:58:10.909363 17774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53be75cf33b74f38884080e1936c65f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53be75cf33b74f38884080e1936c65f1" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 41083 } } }
I20240502 07:58:10.909500 17774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1 [sys.catalog]: This master's current role is: LEADER
I20240502 07:58:10.909353 17775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53be75cf33b74f38884080e1936c65f1. Latest consensus state: current_term: 1 leader_uuid: "53be75cf33b74f38884080e1936c65f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53be75cf33b74f38884080e1936c65f1" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 41083 } } }
I20240502 07:58:10.909631 17775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1 [sys.catalog]: This master's current role is: LEADER
W20240502 07:58:10.912098 17788 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240502 07:58:10.912170 17788 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240502 07:58:10.912292 17786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 07:58:10.913199 17786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 07:58:10.918134 17786 catalog_manager.cc:1347] Generated new cluster ID: 58edb66c039c4468aa9bb59715a5241e
I20240502 07:58:10.918201 17786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 07:58:10.942781 17786 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 07:58:10.943805 17786 catalog_manager.cc:1504] Loading token signing keys...
I20240502 07:58:10.949311 17786 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 53be75cf33b74f38884080e1936c65f1: Generated new TSK 0
I20240502 07:58:10.950204 17786 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 07:58:10.960422 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.129:0
--local_ip_for_outbound_sockets=127.15.129.129
--webserver_interface=127.15.129.129
--webserver_port=0
--tserver_master_addrs=127.15.129.190:41083
--builtin_ntp_servers=127.15.129.148:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 07:58:11.068778 17792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:11.069001 17792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:11.069077 17792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:11.072170 17792 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:11.072255 17792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:11.072364 17792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.129
I20240502 07:58:11.076366 17792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.15.129.129
--webserver_port=0
--tserver_master_addrs=127.15.129.190:41083
--never_fsync=true
--heap_profile_path=/tmp/kudu.17792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.129
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:11.077312 17792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:11.078461 17792 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:11.087905 17800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:11.089190 17797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:11.090302 17798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:11.091630 17792 server_base.cc:1034] running on GCE node
I20240502 07:58:11.092022 17792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:11.092603 17792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:11.093747 17792 hybrid_clock.cc:648] HybridClock initialized: now 1714636691093736 us; error 27 us; skew 500 ppm
I20240502 07:58:11.095597 17792 webserver.cc:460] Webserver started at http://127.15.129.129:38763/ using document root <none> and password file <none>
I20240502 07:58:11.096146 17792 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:11.096204 17792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:11.096383 17792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:11.098059 17792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "3d36406429504d36ab670d84f4464efa"
format_stamp: "Formatted at 2024-05-02 07:58:11 on dist-test-slave-0hx2"
I20240502 07:58:11.098516 17792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "3d36406429504d36ab670d84f4464efa"
format_stamp: "Formatted at 2024-05-02 07:58:11 on dist-test-slave-0hx2"
I20240502 07:58:11.101714 17792 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20240502 07:58:11.103785 17806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:11.104842 17792 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:11.104962 17792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "3d36406429504d36ab670d84f4464efa"
format_stamp: "Formatted at 2024-05-02 07:58:11 on dist-test-slave-0hx2"
I20240502 07:58:11.105057 17792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:11.114789 17792 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:11.115410 17792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:11.115563 17792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:11.116070 17792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:11.116916 17792 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:11.116961 17792 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:11.117005 17792 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:11.117035 17792 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:11.125284 17792 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.129:42343
I20240502 07:58:11.125308 17919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.129:42343 every 8 connection(s)
I20240502 07:58:11.126258 17792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 07:58:11.134286 17920 heartbeater.cc:347] Connected to a master server at 127.15.129.190:41083
I20240502 07:58:11.134476 17920 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:11.134871 17920 heartbeater.cc:511] Master 127.15.129.190:41083 requested a full tablet report, sending...
I20240502 07:58:11.136135 17732 ts_manager.cc:194] Registered new tserver with Master: 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343)
I20240502 07:58:11.136325 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 17792
I20240502 07:58:11.136422 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 07:58:11.137348 17732 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.129:46933
I20240502 07:58:11.138547 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.130:0
--local_ip_for_outbound_sockets=127.15.129.130
--webserver_interface=127.15.129.130
--webserver_port=0
--tserver_master_addrs=127.15.129.190:41083
--builtin_ntp_servers=127.15.129.148:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 07:58:11.244066 17923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:11.244290 17923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:11.244365 17923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:11.247413 17923 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:11.247489 17923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:11.247596 17923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.130
I20240502 07:58:11.251173 17923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.15.129.130
--webserver_port=0
--tserver_master_addrs=127.15.129.190:41083
--never_fsync=true
--heap_profile_path=/tmp/kudu.17923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.130
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:11.252102 17923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:11.253239 17923 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:11.262241 17929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:11.263525 17928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:11.264690 17931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:11.266038 17923 server_base.cc:1034] running on GCE node
I20240502 07:58:11.266459 17923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:11.267046 17923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:11.268198 17923 hybrid_clock.cc:648] HybridClock initialized: now 1714636691268170 us; error 47 us; skew 500 ppm
I20240502 07:58:11.270138 17923 webserver.cc:460] Webserver started at http://127.15.129.130:39689/ using document root <none> and password file <none>
I20240502 07:58:11.270689 17923 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:11.270746 17923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:11.270923 17923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:11.272565 17923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "923d7729977f4edda0aa760e184a8830"
format_stamp: "Formatted at 2024-05-02 07:58:11 on dist-test-slave-0hx2"
I20240502 07:58:11.273018 17923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "923d7729977f4edda0aa760e184a8830"
format_stamp: "Formatted at 2024-05-02 07:58:11 on dist-test-slave-0hx2"
I20240502 07:58:11.276631 17923 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20240502 07:58:11.278934 17937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:11.279944 17923 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:11.280061 17923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "923d7729977f4edda0aa760e184a8830"
format_stamp: "Formatted at 2024-05-02 07:58:11 on dist-test-slave-0hx2"
I20240502 07:58:11.280153 17923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:11.312615 17923 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:11.313269 17923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:11.313452 17923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:11.314023 17923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:11.314898 17923 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:11.314944 17923 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:11.315003 17923 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:11.315028 17923 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:11.323870 17923 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.130:45189
I20240502 07:58:11.324025 18050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.130:45189 every 8 connection(s)
I20240502 07:58:11.325013 17923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 07:58:11.333540 18051 heartbeater.cc:347] Connected to a master server at 127.15.129.190:41083
I20240502 07:58:11.333768 18051 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:11.334148 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 17923
I20240502 07:58:11.334208 18051 heartbeater.cc:511] Master 127.15.129.190:41083 requested a full tablet report, sending...
I20240502 07:58:11.334255 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 07:58:11.335202 17732 ts_manager.cc:194] Registered new tserver with Master: 923d7729977f4edda0aa760e184a8830 (127.15.129.130:45189)
I20240502 07:58:11.335850 17732 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.130:46541
I20240502 07:58:11.336246 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.131:0
--local_ip_for_outbound_sockets=127.15.129.131
--webserver_interface=127.15.129.131
--webserver_port=0
--tserver_master_addrs=127.15.129.190:41083
--builtin_ntp_servers=127.15.129.148:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 07:58:11.444414 18054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:11.444651 18054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:11.444697 18054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:11.447780 18054 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:11.447854 18054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:11.447925 18054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.131
I20240502 07:58:11.451814 18054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.15.129.131
--webserver_port=0
--tserver_master_addrs=127.15.129.190:41083
--never_fsync=true
--heap_profile_path=/tmp/kudu.18054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.131
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:11.452728 18054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:11.453927 18054 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:11.463138 18062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:11.464443 18059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:11.465546 18060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:11.466734 18054 server_base.cc:1034] running on GCE node
I20240502 07:58:11.467098 18054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:11.467639 18054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:11.468808 18054 hybrid_clock.cc:648] HybridClock initialized: now 1714636691468796 us; error 30 us; skew 500 ppm
I20240502 07:58:11.470685 18054 webserver.cc:460] Webserver started at http://127.15.129.131:40647/ using document root <none> and password file <none>
I20240502 07:58:11.471277 18054 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:11.471345 18054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:11.471532 18054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:11.473170 18054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f"
format_stamp: "Formatted at 2024-05-02 07:58:11 on dist-test-slave-0hx2"
I20240502 07:58:11.473685 18054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f"
format_stamp: "Formatted at 2024-05-02 07:58:11 on dist-test-slave-0hx2"
I20240502 07:58:11.476902 18054 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:11.479094 18068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:11.480057 18054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:11.480159 18054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f"
format_stamp: "Formatted at 2024-05-02 07:58:11 on dist-test-slave-0hx2"
I20240502 07:58:11.480245 18054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:11.520936 18054 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:11.521670 18054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:11.521847 18054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:11.522392 18054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:11.523269 18054 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:11.523327 18054 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:11.523391 18054 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:11.523416 18054 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:11.531913 18054 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.131:43267
I20240502 07:58:11.532037 18181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.131:43267 every 8 connection(s)
I20240502 07:58:11.532855 18054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 07:58:11.542050 18182 heartbeater.cc:347] Connected to a master server at 127.15.129.190:41083
I20240502 07:58:11.542321 18182 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:11.542394 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 18054
I20240502 07:58:11.542495 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 07:58:11.542898 18182 heartbeater.cc:511] Master 127.15.129.190:41083 requested a full tablet report, sending...
I20240502 07:58:11.543900 17731 ts_manager.cc:194] Registered new tserver with Master: 9fdc864a7d9349808a6fc7b2cf4eba8f (127.15.129.131:43267)
I20240502 07:58:11.544529 17731 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.131:50117
I20240502 07:58:11.544570 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.132:0
--local_ip_for_outbound_sockets=127.15.129.132
--webserver_interface=127.15.129.132
--webserver_port=0
--tserver_master_addrs=127.15.129.190:41083
--builtin_ntp_servers=127.15.129.148:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 07:58:11.652228 18185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:11.652483 18185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:11.652563 18185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:11.655779 18185 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:11.655865 18185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:11.655959 18185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.132
I20240502 07:58:11.659678 18185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.15.129.132
--webserver_port=0
--tserver_master_addrs=127.15.129.190:41083
--never_fsync=true
--heap_profile_path=/tmp/kudu.18185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.132
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:11.660593 18185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:11.661721 18185 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:11.670600 18190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:11.671489 18191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:11.672827 18193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:11.673949 18185 server_base.cc:1034] running on GCE node
I20240502 07:58:11.674355 18185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:11.674911 18185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:11.676066 18185 hybrid_clock.cc:648] HybridClock initialized: now 1714636691676043 us; error 39 us; skew 500 ppm
I20240502 07:58:11.677942 18185 webserver.cc:460] Webserver started at http://127.15.129.132:41309/ using document root <none> and password file <none>
I20240502 07:58:11.678493 18185 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:11.678550 18185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:11.678723 18185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:11.680382 18185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "3f7795cc262b43cfa8a9252c424b7354"
format_stamp: "Formatted at 2024-05-02 07:58:11 on dist-test-slave-0hx2"
I20240502 07:58:11.680824 18185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "3f7795cc262b43cfa8a9252c424b7354"
format_stamp: "Formatted at 2024-05-02 07:58:11 on dist-test-slave-0hx2"
I20240502 07:58:11.684269 18185 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 07:58:11.686470 18199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:11.687405 18185 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:11.687507 18185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "3f7795cc262b43cfa8a9252c424b7354"
format_stamp: "Formatted at 2024-05-02 07:58:11 on dist-test-slave-0hx2"
I20240502 07:58:11.687590 18185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:11.714310 18185 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:11.714982 18185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:11.715147 18185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:11.715700 18185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:11.716579 18185 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:11.716629 18185 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:11.716687 18185 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:11.716722 18185 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:11.725015 18185 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.132:43973
I20240502 07:58:11.725040 18312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.132:43973 every 8 connection(s)
I20240502 07:58:11.726043 18185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 07:58:11.730624 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 18185
I20240502 07:58:11.730719 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 07:58:11.732846 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.133:0
--local_ip_for_outbound_sockets=127.15.129.133
--webserver_interface=127.15.129.133
--webserver_port=0
--tserver_master_addrs=127.15.129.190:41083
--builtin_ntp_servers=127.15.129.148:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:11.734848 18313 heartbeater.cc:347] Connected to a master server at 127.15.129.190:41083
I20240502 07:58:11.735071 18313 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:11.735509 18313 heartbeater.cc:511] Master 127.15.129.190:41083 requested a full tablet report, sending...
I20240502 07:58:11.736409 17731 ts_manager.cc:194] Registered new tserver with Master: 3f7795cc262b43cfa8a9252c424b7354 (127.15.129.132:43973)
I20240502 07:58:11.736960 17731 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.132:37309
W20240502 07:58:11.845234 18316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:11.845525 18316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:11.845605 18316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:11.848820 18316 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:11.848915 18316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:11.849017 18316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.133
I20240502 07:58:11.852790 18316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.15.129.133
--webserver_port=0
--tserver_master_addrs=127.15.129.190:41083
--never_fsync=true
--heap_profile_path=/tmp/kudu.18316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.133
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:11.853871 18316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:11.855041 18316 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:11.864725 18324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:11.868077 18316 server_base.cc:1034] running on GCE node
W20240502 07:58:11.869423 18321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:11.870292 18322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:11.870762 18316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:11.871405 18316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:11.872560 18316 hybrid_clock.cc:648] HybridClock initialized: now 1714636691872535 us; error 46 us; skew 500 ppm
I20240502 07:58:11.874637 18316 webserver.cc:460] Webserver started at http://127.15.129.133:37597/ using document root <none> and password file <none>
I20240502 07:58:11.875197 18316 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:11.875265 18316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:11.875468 18316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:11.877143 18316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "451db10510a9469b92abc0705d7bf44d"
format_stamp: "Formatted at 2024-05-02 07:58:11 on dist-test-slave-0hx2"
I20240502 07:58:11.877658 18316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "451db10510a9469b92abc0705d7bf44d"
format_stamp: "Formatted at 2024-05-02 07:58:11 on dist-test-slave-0hx2"
I20240502 07:58:11.880937 18316 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 07:58:11.883127 18330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:11.884310 18316 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:11.884420 18316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "451db10510a9469b92abc0705d7bf44d"
format_stamp: "Formatted at 2024-05-02 07:58:11 on dist-test-slave-0hx2"
I20240502 07:58:11.884498 18316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:11.897447 18316 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:11.898200 18316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:11.898382 18316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:11.899008 18316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:11.900038 18316 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:11.900095 18316 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:11.900143 18316 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:11.900171 18316 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:11.909888 18316 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.133:34385
I20240502 07:58:11.909955 18443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.133:34385 every 8 connection(s)
I20240502 07:58:11.910835 18316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 07:58:11.918960 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 18316
I20240502 07:58:11.919100 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 07:58:11.919358 18444 heartbeater.cc:347] Connected to a master server at 127.15.129.190:41083
I20240502 07:58:11.919575 18444 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:11.920063 18444 heartbeater.cc:511] Master 127.15.129.190:41083 requested a full tablet report, sending...
I20240502 07:58:11.920987 17731 ts_manager.cc:194] Registered new tserver with Master: 451db10510a9469b92abc0705d7bf44d (127.15.129.133:34385)
I20240502 07:58:11.921519 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.134:0
--local_ip_for_outbound_sockets=127.15.129.134
--webserver_interface=127.15.129.134
--webserver_port=0
--tserver_master_addrs=127.15.129.190:41083
--builtin_ntp_servers=127.15.129.148:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:11.921790 17731 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.133:46047
W20240502 07:58:12.029552 18447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:12.029783 18447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:12.029858 18447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:12.033097 18447 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:12.033169 18447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:12.033265 18447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.134
I20240502 07:58:12.036947 18447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.134:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.15.129.134
--webserver_port=0
--tserver_master_addrs=127.15.129.190:41083
--never_fsync=true
--heap_profile_path=/tmp/kudu.18447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.134
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:12.037984 18447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:12.039105 18447 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:12.048403 18455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:12.049247 18452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:12.050612 18453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:12.051849 18447 server_base.cc:1034] running on GCE node
I20240502 07:58:12.052477 18447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:12.053102 18447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:12.054263 18447 hybrid_clock.cc:648] HybridClock initialized: now 1714636692054209 us; error 55 us; skew 500 ppm
I20240502 07:58:12.056102 18447 webserver.cc:460] Webserver started at http://127.15.129.134:33535/ using document root <none> and password file <none>
I20240502 07:58:12.056634 18447 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:12.056686 18447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:12.056869 18447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:12.058598 18447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "c815aa98415f4682b2c8429a67e85444"
format_stamp: "Formatted at 2024-05-02 07:58:12 on dist-test-slave-0hx2"
I20240502 07:58:12.059074 18447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "c815aa98415f4682b2c8429a67e85444"
format_stamp: "Formatted at 2024-05-02 07:58:12 on dist-test-slave-0hx2"
I20240502 07:58:12.062424 18447 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:58:12.064610 18461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:12.065651 18447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:58:12.065763 18447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "c815aa98415f4682b2c8429a67e85444"
format_stamp: "Formatted at 2024-05-02 07:58:12 on dist-test-slave-0hx2"
I20240502 07:58:12.065846 18447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:12.085151 18447 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:12.085851 18447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:12.085983 18447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:12.086474 18447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:12.087352 18447 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:12.087399 18447 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:12.087440 18447 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:12.087461 18447 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:12.096012 18447 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.134:34275
I20240502 07:58:12.096041 18574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.134:34275 every 8 connection(s)
I20240502 07:58:12.096940 18447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 07:58:12.097510 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 18447
I20240502 07:58:12.097632 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240502 07:58:12.101199 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.135:0
--local_ip_for_outbound_sockets=127.15.129.135
--webserver_interface=127.15.129.135
--webserver_port=0
--tserver_master_addrs=127.15.129.190:41083
--builtin_ntp_servers=127.15.129.148:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:12.109133 18575 heartbeater.cc:347] Connected to a master server at 127.15.129.190:41083
I20240502 07:58:12.109473 18575 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:12.110167 18575 heartbeater.cc:511] Master 127.15.129.190:41083 requested a full tablet report, sending...
I20240502 07:58:12.111342 17731 ts_manager.cc:194] Registered new tserver with Master: c815aa98415f4682b2c8429a67e85444 (127.15.129.134:34275)
I20240502 07:58:12.111938 17731 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.134:58239
I20240502 07:58:12.139559 17920 heartbeater.cc:503] Master 127.15.129.190:41083 was elected leader, sending a full tablet report...
W20240502 07:58:12.215386 18578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:12.215644 18578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:12.215719 18578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:12.218930 18578 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:12.219013 18578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:12.219107 18578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.135
I20240502 07:58:12.222913 18578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.135:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.15.129.135
--webserver_port=0
--tserver_master_addrs=127.15.129.190:41083
--never_fsync=true
--heap_profile_path=/tmp/kudu.18578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.135
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:12.223886 18578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:12.225013 18578 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:12.234026 18583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:12.234918 18586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:12.236253 18584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:12.237397 18578 server_base.cc:1034] running on GCE node
I20240502 07:58:12.237797 18578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:12.238373 18578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:12.239545 18578 hybrid_clock.cc:648] HybridClock initialized: now 1714636692239524 us; error 38 us; skew 500 ppm
I20240502 07:58:12.241457 18578 webserver.cc:460] Webserver started at http://127.15.129.135:38639/ using document root <none> and password file <none>
I20240502 07:58:12.242017 18578 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:12.242094 18578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:12.242309 18578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:12.244027 18578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "d87eea2501954fca83327b9408429b66"
format_stamp: "Formatted at 2024-05-02 07:58:12 on dist-test-slave-0hx2"
I20240502 07:58:12.244498 18578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "d87eea2501954fca83327b9408429b66"
format_stamp: "Formatted at 2024-05-02 07:58:12 on dist-test-slave-0hx2"
I20240502 07:58:12.247782 18578 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20240502 07:58:12.250059 18592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:12.251161 18578 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:12.251284 18578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "d87eea2501954fca83327b9408429b66"
format_stamp: "Formatted at 2024-05-02 07:58:12 on dist-test-slave-0hx2"
I20240502 07:58:12.251374 18578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:12.262333 18578 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:12.262975 18578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:12.263127 18578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:12.263660 18578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:12.264540 18578 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:12.264590 18578 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:12.264650 18578 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:12.264676 18578 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:12.273355 18578 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.135:37247
I20240502 07:58:12.273376 18705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.135:37247 every 8 connection(s)
I20240502 07:58:12.274333 18578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 07:58:12.277598 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 18578
I20240502 07:58:12.277709 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240502 07:58:12.282971 18706 heartbeater.cc:347] Connected to a master server at 127.15.129.190:41083
I20240502 07:58:12.283185 18706 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:12.283641 18706 heartbeater.cc:511] Master 127.15.129.190:41083 requested a full tablet report, sending...
I20240502 07:58:12.284637 17731 ts_manager.cc:194] Registered new tserver with Master: d87eea2501954fca83327b9408429b66 (127.15.129.135:37247)
I20240502 07:58:12.285244 17731 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.135:33235
I20240502 07:58:12.292011 15878 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240502 07:58:12.300837 17731 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43302:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20240502 07:58:12.317680 18378 tablet_service.cc:1467] Processing CreateTablet for tablet 41a4e8bf2f494f159b25a8437f7bdeec (DEFAULT_TABLE table=TestTable [id=68382c14994c49889189f6d7102699b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:12.318527 18378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41a4e8bf2f494f159b25a8437f7bdeec. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:12.323559 18720 tablet_bootstrap.cc:492] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d: Bootstrap starting.
I20240502 07:58:12.325785 18720 tablet_bootstrap.cc:654] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:12.326567 18720 log.cc:824] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:12.328984 18720 tablet_bootstrap.cc:492] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d: No bootstrap required, opened a new log
I20240502 07:58:12.329157 18720 ts_tablet_manager.cc:1367] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240502 07:58:12.332016 18720 raft_consensus.cc:348] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } }
I20240502 07:58:12.332186 18720 raft_consensus.cc:374] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:12.332252 18720 raft_consensus.cc:729] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 451db10510a9469b92abc0705d7bf44d, State: Initialized, Role: FOLLOWER
I20240502 07:58:12.332665 18720 consensus_queue.cc:260] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } }
I20240502 07:58:12.332789 18720 raft_consensus.cc:388] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:12.332849 18720 raft_consensus.cc:482] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:12.332963 18720 raft_consensus.cc:3037] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:12.333938 18720 raft_consensus.cc:504] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } }
I20240502 07:58:12.334261 18720 leader_election.cc:302] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 451db10510a9469b92abc0705d7bf44d; no voters:
I20240502 07:58:12.334542 18720 leader_election.cc:288] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:12.334635 18722 raft_consensus.cc:2781] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:12.334862 18722 raft_consensus.cc:686] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [term 1 LEADER]: Becoming Leader. State: Replica: 451db10510a9469b92abc0705d7bf44d, State: Running, Role: LEADER
I20240502 07:58:12.335165 18722 consensus_queue.cc:237] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } }
I20240502 07:58:12.335239 18444 heartbeater.cc:503] Master 127.15.129.190:41083 was elected leader, sending a full tablet report...
I20240502 07:58:12.335400 18720 ts_tablet_manager.cc:1398] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20240502 07:58:12.337718 18051 heartbeater.cc:503] Master 127.15.129.190:41083 was elected leader, sending a full tablet report...
I20240502 07:58:12.338222 17731 catalog_manager.cc:5484] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d reported cstate change: term changed from 0 to 1, leader changed from <none> to 451db10510a9469b92abc0705d7bf44d (127.15.129.133). New cstate: current_term: 1 leader_uuid: "451db10510a9469b92abc0705d7bf44d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } } }
I20240502 07:58:12.343982 15878 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20240502 07:58:12.345256 15878 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 3d36406429504d36ab670d84f4464efa to finish bootstrapping
I20240502 07:58:12.350952 15878 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 923d7729977f4edda0aa760e184a8830 to finish bootstrapping
I20240502 07:58:12.356791 15878 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 9fdc864a7d9349808a6fc7b2cf4eba8f to finish bootstrapping
I20240502 07:58:12.362129 15878 ts_itest-base.cc:233] Waiting for 0 tablets on tserver 3f7795cc262b43cfa8a9252c424b7354 to finish bootstrapping
I20240502 07:58:12.367041 15878 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 451db10510a9469b92abc0705d7bf44d to finish bootstrapping
I20240502 07:58:12.371677 15878 ts_itest-base.cc:233] Waiting for 0 tablets on tserver c815aa98415f4682b2c8429a67e85444 to finish bootstrapping
I20240502 07:58:12.377085 15878 ts_itest-base.cc:233] Waiting for 0 tablets on tserver d87eea2501954fca83327b9408429b66 to finish bootstrapping
I20240502 07:58:12.382221 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 18054
I20240502 07:58:12.387887 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 18185
I20240502 07:58:12.393167 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 18316
I20240502 07:58:12.398761 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 18447
I20240502 07:58:12.405736 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 18578
I20240502 07:58:16.162863 17732 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43302:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:16.175580 17854 tablet_service.cc:1467] Processing CreateTablet for tablet 4fb3087aff7d40eb960135010402371e (DEFAULT_TABLE table=rebalance_test_table_0 [id=cb32abd9e63d47ffa9ab6f1229f6279c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:16.175577 17985 tablet_service.cc:1467] Processing CreateTablet for tablet a7a2522085124c63a938200b3e925cdb (DEFAULT_TABLE table=rebalance_test_table_0 [id=cb32abd9e63d47ffa9ab6f1229f6279c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:16.175578 17853 tablet_service.cc:1467] Processing CreateTablet for tablet 4609595bd09d4893b2575ca6a2740896 (DEFAULT_TABLE table=rebalance_test_table_0 [id=cb32abd9e63d47ffa9ab6f1229f6279c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:16.176573 17853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4609595bd09d4893b2575ca6a2740896. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:16.176573 17985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7a2522085124c63a938200b3e925cdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:16.177003 17854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fb3087aff7d40eb960135010402371e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:16.181946 18742 tablet_bootstrap.cc:492] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa: Bootstrap starting.
I20240502 07:58:16.182135 18743 tablet_bootstrap.cc:492] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830: Bootstrap starting.
I20240502 07:58:16.183943 18742 tablet_bootstrap.cc:654] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:16.183945 18743 tablet_bootstrap.cc:654] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:16.184576 18743 log.cc:824] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:16.184595 18742 log.cc:824] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:16.186038 18743 tablet_bootstrap.cc:492] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830: No bootstrap required, opened a new log
I20240502 07:58:16.186036 18742 tablet_bootstrap.cc:492] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa: No bootstrap required, opened a new log
I20240502 07:58:16.186187 18743 ts_tablet_manager.cc:1367] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 07:58:16.186192 18742 ts_tablet_manager.cc:1367] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 07:58:16.189038 18743 raft_consensus.cc:348] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.189038 18742 raft_consensus.cc:348] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.189208 18742 raft_consensus.cc:374] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:16.189210 18743 raft_consensus.cc:374] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:16.189239 18742 raft_consensus.cc:729] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Initialized, Role: FOLLOWER
I20240502 07:58:16.189261 18743 raft_consensus.cc:729] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Initialized, Role: FOLLOWER
I20240502 07:58:16.189642 18743 consensus_queue.cc:260] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.189646 18742 consensus_queue.cc:260] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.189760 18742 raft_consensus.cc:388] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:16.189762 18743 raft_consensus.cc:388] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:16.189826 18742 raft_consensus.cc:482] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:16.189827 18743 raft_consensus.cc:482] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:16.189927 18742 raft_consensus.cc:3037] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:16.189929 18743 raft_consensus.cc:3037] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:16.190672 18742 raft_consensus.cc:504] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.190680 18743 raft_consensus.cc:504] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.190972 18742 leader_election.cc:302] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d36406429504d36ab670d84f4464efa; no voters:
I20240502 07:58:16.190976 18743 leader_election.cc:302] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 923d7729977f4edda0aa760e184a8830; no voters:
I20240502 07:58:16.191258 18743 leader_election.cc:288] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:16.191262 18742 leader_election.cc:288] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:16.191352 18747 raft_consensus.cc:2781] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:16.191557 18749 raft_consensus.cc:2781] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:16.191569 18747 raft_consensus.cc:686] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Becoming Leader. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Running, Role: LEADER
I20240502 07:58:16.191768 18749 raft_consensus.cc:686] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Becoming Leader. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Running, Role: LEADER
I20240502 07:58:16.191824 18747 consensus_queue.cc:237] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.192052 18743 ts_tablet_manager.cc:1398] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240502 07:58:16.192044 18742 ts_tablet_manager.cc:1398] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240502 07:58:16.192144 18749 consensus_queue.cc:237] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.192684 18742 tablet_bootstrap.cc:492] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa: Bootstrap starting.
I20240502 07:58:16.194368 18742 tablet_bootstrap.cc:654] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:16.194411 17732 catalog_manager.cc:5484] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d36406429504d36ab670d84f4464efa (127.15.129.129). New cstate: current_term: 1 leader_uuid: "3d36406429504d36ab670d84f4464efa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } }
I20240502 07:58:16.195778 17731 catalog_manager.cc:5484] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 reported cstate change: term changed from 0 to 1, leader changed from <none> to 923d7729977f4edda0aa760e184a8830 (127.15.129.130). New cstate: current_term: 1 leader_uuid: "923d7729977f4edda0aa760e184a8830" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } }
I20240502 07:58:16.195961 18742 tablet_bootstrap.cc:492] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa: No bootstrap required, opened a new log
I20240502 07:58:16.196033 18742 ts_tablet_manager.cc:1367] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:16.196568 18742 raft_consensus.cc:348] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.196673 18742 raft_consensus.cc:374] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:16.196722 18742 raft_consensus.cc:729] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Initialized, Role: FOLLOWER
I20240502 07:58:16.196846 18742 consensus_queue.cc:260] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.196933 18742 raft_consensus.cc:388] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:16.196969 18742 raft_consensus.cc:482] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:16.197008 18742 raft_consensus.cc:3037] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:16.197789 18742 raft_consensus.cc:504] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.197916 18742 leader_election.cc:302] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d36406429504d36ab670d84f4464efa; no voters:
I20240502 07:58:16.198017 18742 leader_election.cc:288] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:16.198083 18750 raft_consensus.cc:2781] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:16.198173 18750 raft_consensus.cc:686] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Becoming Leader. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Running, Role: LEADER
I20240502 07:58:16.198215 18742 ts_tablet_manager.cc:1398] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:16.198282 18750 consensus_queue.cc:237] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.199949 17731 catalog_manager.cc:5484] T 4609595bd09d4893b2575ca6a2740896 P 3d36406429504d36ab670d84f4464efa reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d36406429504d36ab670d84f4464efa (127.15.129.129). New cstate: current_term: 1 leader_uuid: "3d36406429504d36ab670d84f4464efa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } }
W20240502 07:58:16.315814 18758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:16.316066 18758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:16.319186 18758 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
INSERT report
rows total: 2000
time total: 88.2167 ms
time per row: 0.0441083 ms
I20240502 07:58:16.449618 17729 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43302:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:16.455557 17985 tablet_service.cc:1467] Processing CreateTablet for tablet 9d165c26beff4f3dacf61ca92fdec35f (DEFAULT_TABLE table=rebalance_test_table_1 [id=660ea085db6f4a49a3d523727eb2b72f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:16.455664 17984 tablet_service.cc:1467] Processing CreateTablet for tablet 3f6c4761aef54c538693f1d3a4d80d10 (DEFAULT_TABLE table=rebalance_test_table_1 [id=660ea085db6f4a49a3d523727eb2b72f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:16.455622 17854 tablet_service.cc:1467] Processing CreateTablet for tablet f49ef9b8bef141a3bc00ec7bf7c1ac67 (DEFAULT_TABLE table=rebalance_test_table_1 [id=660ea085db6f4a49a3d523727eb2b72f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:16.455911 17985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d165c26beff4f3dacf61ca92fdec35f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:16.455930 17854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f49ef9b8bef141a3bc00ec7bf7c1ac67. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:16.456151 17984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f6c4761aef54c538693f1d3a4d80d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:16.457747 18742 tablet_bootstrap.cc:492] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa: Bootstrap starting.
I20240502 07:58:16.458201 18743 tablet_bootstrap.cc:492] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830: Bootstrap starting.
I20240502 07:58:16.458654 18742 tablet_bootstrap.cc:654] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:16.458956 18743 tablet_bootstrap.cc:654] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:16.459801 18742 tablet_bootstrap.cc:492] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa: No bootstrap required, opened a new log
I20240502 07:58:16.459903 18742 ts_tablet_manager.cc:1367] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:16.459973 18743 tablet_bootstrap.cc:492] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830: No bootstrap required, opened a new log
I20240502 07:58:16.460059 18743 ts_tablet_manager.cc:1367] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:16.460423 18742 raft_consensus.cc:348] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.460529 18742 raft_consensus.cc:374] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:16.460563 18742 raft_consensus.cc:729] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Initialized, Role: FOLLOWER
I20240502 07:58:16.460592 18743 raft_consensus.cc:348] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.460673 18743 raft_consensus.cc:374] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:16.460702 18743 raft_consensus.cc:729] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Initialized, Role: FOLLOWER
I20240502 07:58:16.460701 18742 consensus_queue.cc:260] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.460819 18743 consensus_queue.cc:260] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.460790 18742 raft_consensus.cc:388] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:16.460898 18743 raft_consensus.cc:388] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:16.460928 18743 raft_consensus.cc:482] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:16.460906 18742 raft_consensus.cc:482] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:16.460958 18743 raft_consensus.cc:3037] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:16.460992 18742 raft_consensus.cc:3037] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:16.461807 18743 raft_consensus.cc:504] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.461815 18742 raft_consensus.cc:504] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.461920 18743 leader_election.cc:302] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 923d7729977f4edda0aa760e184a8830; no voters:
I20240502 07:58:16.461951 18742 leader_election.cc:302] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d36406429504d36ab670d84f4464efa; no voters:
I20240502 07:58:16.462049 18743 leader_election.cc:288] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:16.462133 18742 leader_election.cc:288] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:16.462116 18751 raft_consensus.cc:2781] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:16.462268 18743 ts_tablet_manager.cc:1398] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:16.462355 18742 ts_tablet_manager.cc:1398] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 07:58:16.462388 18743 tablet_bootstrap.cc:492] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830: Bootstrap starting.
I20240502 07:58:16.462512 18748 raft_consensus.cc:2781] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:16.462576 18748 raft_consensus.cc:686] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Becoming Leader. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Running, Role: LEADER
I20240502 07:58:16.462719 18748 consensus_queue.cc:237] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.463107 18751 raft_consensus.cc:686] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Becoming Leader. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Running, Role: LEADER
I20240502 07:58:16.463258 18751 consensus_queue.cc:237] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.464954 17729 catalog_manager.cc:5484] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d36406429504d36ab670d84f4464efa (127.15.129.129). New cstate: current_term: 1 leader_uuid: "3d36406429504d36ab670d84f4464efa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } }
I20240502 07:58:16.465107 17728 catalog_manager.cc:5484] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 reported cstate change: term changed from 0 to 1, leader changed from <none> to 923d7729977f4edda0aa760e184a8830 (127.15.129.130). New cstate: current_term: 1 leader_uuid: "923d7729977f4edda0aa760e184a8830" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } }
I20240502 07:58:16.463140 18743 tablet_bootstrap.cc:654] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:16.466667 18743 tablet_bootstrap.cc:492] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830: No bootstrap required, opened a new log
I20240502 07:58:16.466758 18743 ts_tablet_manager.cc:1367] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:16.467237 18743 raft_consensus.cc:348] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.467336 18743 raft_consensus.cc:374] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:16.467372 18743 raft_consensus.cc:729] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Initialized, Role: FOLLOWER
I20240502 07:58:16.467525 18743 consensus_queue.cc:260] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.467610 18743 raft_consensus.cc:388] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:16.467670 18743 raft_consensus.cc:482] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:16.467758 18743 raft_consensus.cc:3037] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:16.469082 18743 raft_consensus.cc:504] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.469288 18743 leader_election.cc:302] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 923d7729977f4edda0aa760e184a8830; no voters:
I20240502 07:58:16.469444 18743 leader_election.cc:288] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:16.469512 18749 raft_consensus.cc:2781] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:16.469633 18749 raft_consensus.cc:686] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Becoming Leader. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Running, Role: LEADER
I20240502 07:58:16.469810 18749 consensus_queue.cc:237] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.469913 18743 ts_tablet_manager.cc:1398] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240502 07:58:16.471179 17730 catalog_manager.cc:5484] T 9d165c26beff4f3dacf61ca92fdec35f P 923d7729977f4edda0aa760e184a8830 reported cstate change: term changed from 0 to 1, leader changed from <none> to 923d7729977f4edda0aa760e184a8830 (127.15.129.130). New cstate: current_term: 1 leader_uuid: "923d7729977f4edda0aa760e184a8830" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } }
W20240502 07:58:16.584934 18800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:16.585222 18800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:16.588279 18800 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
INSERT report
rows total: 2000
time total: 82.0411 ms
time per row: 0.0410205 ms
I20240502 07:58:16.713840 17730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43302:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:16.719161 17985 tablet_service.cc:1467] Processing CreateTablet for tablet 4ae9214f8e5a4113b339c78cf80b1ed8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=46e1dc9f151748c996c37174bb27644c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:16.719157 17854 tablet_service.cc:1467] Processing CreateTablet for tablet 334047bb4be641e0aad4f6405e067468 (DEFAULT_TABLE table=rebalance_test_table_2 [id=46e1dc9f151748c996c37174bb27644c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:16.719173 17853 tablet_service.cc:1467] Processing CreateTablet for tablet 62ba9d55df884c568ea6285701011ceb (DEFAULT_TABLE table=rebalance_test_table_2 [id=46e1dc9f151748c996c37174bb27644c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:16.719470 17854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 334047bb4be641e0aad4f6405e067468. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:16.719470 17985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ae9214f8e5a4113b339c78cf80b1ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:16.719594 17853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62ba9d55df884c568ea6285701011ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:16.721179 18742 tablet_bootstrap.cc:492] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa: Bootstrap starting.
I20240502 07:58:16.721493 18743 tablet_bootstrap.cc:492] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830: Bootstrap starting.
I20240502 07:58:16.722113 18742 tablet_bootstrap.cc:654] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:16.722262 18743 tablet_bootstrap.cc:654] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:16.723162 18743 tablet_bootstrap.cc:492] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830: No bootstrap required, opened a new log
I20240502 07:58:16.723232 18743 ts_tablet_manager.cc:1367] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:16.723562 18743 raft_consensus.cc:348] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.723646 18743 raft_consensus.cc:374] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:16.723677 18743 raft_consensus.cc:729] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Initialized, Role: FOLLOWER
I20240502 07:58:16.723764 18743 consensus_queue.cc:260] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.723819 18743 raft_consensus.cc:388] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:16.723847 18743 raft_consensus.cc:482] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:16.723879 18743 raft_consensus.cc:3037] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:16.724467 18743 raft_consensus.cc:504] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.724576 18743 leader_election.cc:302] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 923d7729977f4edda0aa760e184a8830; no voters:
I20240502 07:58:16.725025 18742 tablet_bootstrap.cc:492] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa: No bootstrap required, opened a new log
I20240502 07:58:16.725117 18742 ts_tablet_manager.cc:1367] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:16.725421 18743 leader_election.cc:288] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:16.725464 18749 raft_consensus.cc:2781] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:16.725641 18749 raft_consensus.cc:686] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Becoming Leader. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Running, Role: LEADER
I20240502 07:58:16.725605 18742 raft_consensus.cc:348] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.725693 18743 ts_tablet_manager.cc:1398] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:16.725694 18742 raft_consensus.cc:374] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:16.725728 18742 raft_consensus.cc:729] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Initialized, Role: FOLLOWER
I20240502 07:58:16.725757 18749 consensus_queue.cc:237] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.725852 18742 consensus_queue.cc:260] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.725936 18742 raft_consensus.cc:388] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:16.725973 18742 raft_consensus.cc:482] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:16.726019 18742 raft_consensus.cc:3037] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:16.726907 18742 raft_consensus.cc:504] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.727041 18742 leader_election.cc:302] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d36406429504d36ab670d84f4464efa; no voters:
I20240502 07:58:16.727133 18742 leader_election.cc:288] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:16.727069 17730 catalog_manager.cc:5484] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 reported cstate change: term changed from 0 to 1, leader changed from <none> to 923d7729977f4edda0aa760e184a8830 (127.15.129.130). New cstate: current_term: 1 leader_uuid: "923d7729977f4edda0aa760e184a8830" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } }
I20240502 07:58:16.727192 18750 raft_consensus.cc:2781] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:16.727300 18750 raft_consensus.cc:686] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Becoming Leader. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Running, Role: LEADER
I20240502 07:58:16.727419 18750 consensus_queue.cc:237] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.727630 18742 ts_tablet_manager.cc:1398] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240502 07:58:16.728271 18742 tablet_bootstrap.cc:492] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa: Bootstrap starting.
I20240502 07:58:16.728586 17723 catalog_manager.cc:5484] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d36406429504d36ab670d84f4464efa (127.15.129.129). New cstate: current_term: 1 leader_uuid: "3d36406429504d36ab670d84f4464efa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } }
I20240502 07:58:16.729806 18742 tablet_bootstrap.cc:654] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:16.731561 18742 tablet_bootstrap.cc:492] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa: No bootstrap required, opened a new log
I20240502 07:58:16.731631 18742 ts_tablet_manager.cc:1367] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:16.731982 18742 raft_consensus.cc:348] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.732084 18742 raft_consensus.cc:374] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:16.732120 18742 raft_consensus.cc:729] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Initialized, Role: FOLLOWER
I20240502 07:58:16.732283 18742 consensus_queue.cc:260] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.732371 18742 raft_consensus.cc:388] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:16.732417 18742 raft_consensus.cc:482] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:16.732463 18742 raft_consensus.cc:3037] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:16.733091 18742 raft_consensus.cc:504] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.733203 18742 leader_election.cc:302] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d36406429504d36ab670d84f4464efa; no voters:
I20240502 07:58:16.733274 18742 leader_election.cc:288] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:16.733353 18750 raft_consensus.cc:2781] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:16.733439 18750 raft_consensus.cc:686] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Becoming Leader. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Running, Role: LEADER
I20240502 07:58:16.733544 18750 consensus_queue.cc:237] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.733582 18742 ts_tablet_manager.cc:1398] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:16.734679 17723 catalog_manager.cc:5484] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d36406429504d36ab670d84f4464efa (127.15.129.129). New cstate: current_term: 1 leader_uuid: "3d36406429504d36ab670d84f4464efa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } }
W20240502 07:58:16.845904 18836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:16.846155 18836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:16.849167 18836 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
INSERT report
rows total: 2000
time total: 60.2518 ms
time per row: 0.0301259 ms
I20240502 07:58:16.949144 17730 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43302:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:16.954694 17985 tablet_service.cc:1467] Processing CreateTablet for tablet 706476e790b54b2690ea892fb6850046 (DEFAULT_TABLE table=rebalance_test_table_3 [id=30cbd54b96274072a006ed7c32764da1]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:16.954758 17984 tablet_service.cc:1467] Processing CreateTablet for tablet e82159be5de549fe9ed93db5ef6262fc (DEFAULT_TABLE table=rebalance_test_table_3 [id=30cbd54b96274072a006ed7c32764da1]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:16.955050 17985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 706476e790b54b2690ea892fb6850046. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:16.955014 17854 tablet_service.cc:1467] Processing CreateTablet for tablet bab58d3c92774b1ab9fa77e100db9073 (DEFAULT_TABLE table=rebalance_test_table_3 [id=30cbd54b96274072a006ed7c32764da1]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:16.955180 17984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e82159be5de549fe9ed93db5ef6262fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:16.955282 17854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bab58d3c92774b1ab9fa77e100db9073. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:16.956727 18743 tablet_bootstrap.cc:492] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830: Bootstrap starting.
I20240502 07:58:16.957449 18742 tablet_bootstrap.cc:492] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa: Bootstrap starting.
I20240502 07:58:16.957973 18743 tablet_bootstrap.cc:654] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:16.958295 18742 tablet_bootstrap.cc:654] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:16.959097 18743 tablet_bootstrap.cc:492] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830: No bootstrap required, opened a new log
I20240502 07:58:16.959182 18743 ts_tablet_manager.cc:1367] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:16.959326 18742 tablet_bootstrap.cc:492] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa: No bootstrap required, opened a new log
I20240502 07:58:16.959398 18742 ts_tablet_manager.cc:1367] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:16.959717 18743 raft_consensus.cc:348] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.959806 18743 raft_consensus.cc:374] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:16.959774 18742 raft_consensus.cc:348] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.959842 18743 raft_consensus.cc:729] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Initialized, Role: FOLLOWER
I20240502 07:58:16.959892 18742 raft_consensus.cc:374] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:16.959926 18742 raft_consensus.cc:729] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Initialized, Role: FOLLOWER
I20240502 07:58:16.959978 18743 consensus_queue.cc:260] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.960065 18743 raft_consensus.cc:388] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:16.960064 18742 consensus_queue.cc:260] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.960120 18743 raft_consensus.cc:482] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:16.960148 18742 raft_consensus.cc:388] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:16.960180 18743 raft_consensus.cc:3037] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:16.960196 18742 raft_consensus.cc:482] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:16.960234 18742 raft_consensus.cc:3037] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:16.960937 18742 raft_consensus.cc:504] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.960999 18743 raft_consensus.cc:504] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.961033 18742 leader_election.cc:302] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d36406429504d36ab670d84f4464efa; no voters:
I20240502 07:58:16.961099 18742 leader_election.cc:288] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:16.961102 18743 leader_election.cc:302] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 923d7729977f4edda0aa760e184a8830; no voters:
I20240502 07:58:16.961192 18743 leader_election.cc:288] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:16.961190 18748 raft_consensus.cc:2781] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:16.961248 18751 raft_consensus.cc:2781] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:16.961280 18748 raft_consensus.cc:686] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Becoming Leader. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Running, Role: LEADER
I20240502 07:58:16.961447 18751 raft_consensus.cc:686] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Becoming Leader. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Running, Role: LEADER
I20240502 07:58:16.961434 18748 consensus_queue.cc:237] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:16.961517 18743 ts_tablet_manager.cc:1398] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20240502 07:58:16.961553 18751 consensus_queue.cc:237] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.961683 18743 tablet_bootstrap.cc:492] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830: Bootstrap starting.
I20240502 07:58:16.962639 18743 tablet_bootstrap.cc:654] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:16.962805 17730 catalog_manager.cc:5484] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d36406429504d36ab670d84f4464efa (127.15.129.129). New cstate: current_term: 1 leader_uuid: "3d36406429504d36ab670d84f4464efa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } }
I20240502 07:58:16.963740 18742 ts_tablet_manager.cc:1398] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:16.964200 18743 tablet_bootstrap.cc:492] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830: No bootstrap required, opened a new log
I20240502 07:58:16.964290 18743 ts_tablet_manager.cc:1367] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:16.964219 17724 catalog_manager.cc:5484] T 706476e790b54b2690ea892fb6850046 P 923d7729977f4edda0aa760e184a8830 reported cstate change: term changed from 0 to 1, leader changed from <none> to 923d7729977f4edda0aa760e184a8830 (127.15.129.130). New cstate: current_term: 1 leader_uuid: "923d7729977f4edda0aa760e184a8830" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } }
I20240502 07:58:16.964708 18743 raft_consensus.cc:348] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.964789 18743 raft_consensus.cc:374] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:16.964812 18743 raft_consensus.cc:729] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Initialized, Role: FOLLOWER
I20240502 07:58:16.964918 18743 consensus_queue.cc:260] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.964988 18743 raft_consensus.cc:388] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:16.965032 18743 raft_consensus.cc:482] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:16.965080 18743 raft_consensus.cc:3037] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:16.966004 18743 raft_consensus.cc:504] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.966128 18743 leader_election.cc:302] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 923d7729977f4edda0aa760e184a8830; no voters:
I20240502 07:58:16.966233 18743 leader_election.cc:288] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:16.966270 18749 raft_consensus.cc:2781] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:16.966361 18749 raft_consensus.cc:686] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Becoming Leader. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Running, Role: LEADER
I20240502 07:58:16.966475 18749 consensus_queue.cc:237] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:16.966670 18743 ts_tablet_manager.cc:1398] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:16.968427 17730 catalog_manager.cc:5484] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 reported cstate change: term changed from 0 to 1, leader changed from <none> to 923d7729977f4edda0aa760e184a8830 (127.15.129.130). New cstate: current_term: 1 leader_uuid: "923d7729977f4edda0aa760e184a8830" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } }
W20240502 07:58:17.079504 18868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:17.079758 18868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:17.082785 18868 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
INSERT report
rows total: 2000
time total: 72.1444 ms
time per row: 0.0360722 ms
I20240502 07:58:17.197191 17727 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43302:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:17.202950 17854 tablet_service.cc:1467] Processing CreateTablet for tablet 323b33d450384c90b13a1ac565f411a4 (DEFAULT_TABLE table=rebalance_test_table_4 [id=aa24b74e8feb4d10b47ce377487513ca]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:17.203119 17853 tablet_service.cc:1467] Processing CreateTablet for tablet 1042d2331b5a425eb8e6721cb5c4b894 (DEFAULT_TABLE table=rebalance_test_table_4 [id=aa24b74e8feb4d10b47ce377487513ca]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:17.203279 17854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 323b33d450384c90b13a1ac565f411a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:17.203253 17984 tablet_service.cc:1467] Processing CreateTablet for tablet 04638eb3c88040f7b40f7f6ec6989e06 (DEFAULT_TABLE table=rebalance_test_table_4 [id=aa24b74e8feb4d10b47ce377487513ca]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:17.203469 17853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1042d2331b5a425eb8e6721cb5c4b894. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:17.203506 17984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04638eb3c88040f7b40f7f6ec6989e06. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:17.204980 18742 tablet_bootstrap.cc:492] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa: Bootstrap starting.
I20240502 07:58:17.205403 18743 tablet_bootstrap.cc:492] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830: Bootstrap starting.
I20240502 07:58:17.205914 18742 tablet_bootstrap.cc:654] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:17.206280 18743 tablet_bootstrap.cc:654] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:17.207012 18742 tablet_bootstrap.cc:492] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa: No bootstrap required, opened a new log
I20240502 07:58:17.207103 18742 ts_tablet_manager.cc:1367] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:17.207279 18743 tablet_bootstrap.cc:492] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830: No bootstrap required, opened a new log
I20240502 07:58:17.207358 18743 ts_tablet_manager.cc:1367] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:17.207571 18742 raft_consensus.cc:348] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:17.207670 18742 raft_consensus.cc:374] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:17.207701 18742 raft_consensus.cc:729] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Initialized, Role: FOLLOWER
I20240502 07:58:17.207762 18743 raft_consensus.cc:348] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:17.207837 18743 raft_consensus.cc:374] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:17.207867 18743 raft_consensus.cc:729] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Initialized, Role: FOLLOWER
I20240502 07:58:17.207830 18742 consensus_queue.cc:260] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:17.207912 18742 raft_consensus.cc:388] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:17.207948 18742 raft_consensus.cc:482] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:17.207988 18742 raft_consensus.cc:3037] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:17.207975 18743 consensus_queue.cc:260] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:17.208060 18743 raft_consensus.cc:388] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:17.208096 18743 raft_consensus.cc:482] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:17.208174 18743 raft_consensus.cc:3037] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:17.208868 18742 raft_consensus.cc:504] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:17.209008 18742 leader_election.cc:302] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d36406429504d36ab670d84f4464efa; no voters:
I20240502 07:58:17.209007 18743 raft_consensus.cc:504] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:17.209097 18742 leader_election.cc:288] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:17.209115 18743 leader_election.cc:302] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 923d7729977f4edda0aa760e184a8830; no voters:
I20240502 07:58:17.209141 18747 raft_consensus.cc:2781] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:17.209244 18747 raft_consensus.cc:686] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Becoming Leader. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Running, Role: LEADER
I20240502 07:58:17.209273 18831 raft_consensus.cc:2781] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:17.209374 18831 raft_consensus.cc:686] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Becoming Leader. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Running, Role: LEADER
I20240502 07:58:17.209424 18747 consensus_queue.cc:237] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:17.209470 18831 consensus_queue.cc:237] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:17.210289 18742 ts_tablet_manager.cc:1398] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240502 07:58:17.210479 18742 tablet_bootstrap.cc:492] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa: Bootstrap starting.
I20240502 07:58:17.210295 18743 leader_election.cc:288] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:17.210738 18743 ts_tablet_manager.cc:1398] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:17.210932 17727 catalog_manager.cc:5484] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 reported cstate change: term changed from 0 to 1, leader changed from <none> to 923d7729977f4edda0aa760e184a8830 (127.15.129.130). New cstate: current_term: 1 leader_uuid: "923d7729977f4edda0aa760e184a8830" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } }
I20240502 07:58:17.210944 17732 catalog_manager.cc:5484] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d36406429504d36ab670d84f4464efa (127.15.129.129). New cstate: current_term: 1 leader_uuid: "3d36406429504d36ab670d84f4464efa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } }
I20240502 07:58:17.211373 18742 tablet_bootstrap.cc:654] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:17.212656 18742 tablet_bootstrap.cc:492] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa: No bootstrap required, opened a new log
I20240502 07:58:17.212733 18742 ts_tablet_manager.cc:1367] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:17.213174 18742 raft_consensus.cc:348] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:17.213279 18742 raft_consensus.cc:374] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:17.213351 18742 raft_consensus.cc:729] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Initialized, Role: FOLLOWER
I20240502 07:58:17.213480 18742 consensus_queue.cc:260] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:17.213567 18742 raft_consensus.cc:388] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:17.213599 18742 raft_consensus.cc:482] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:17.213635 18742 raft_consensus.cc:3037] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:17.214448 18742 raft_consensus.cc:504] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:17.214563 18742 leader_election.cc:302] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d36406429504d36ab670d84f4464efa; no voters:
I20240502 07:58:17.214663 18742 leader_election.cc:288] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:17.214722 18747 raft_consensus.cc:2781] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:17.214833 18747 raft_consensus.cc:686] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Becoming Leader. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Running, Role: LEADER
I20240502 07:58:17.214879 18742 ts_tablet_manager.cc:1398] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:17.214942 18747 consensus_queue.cc:237] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:17.217168 17732 catalog_manager.cc:5484] T 1042d2331b5a425eb8e6721cb5c4b894 P 3d36406429504d36ab670d84f4464efa reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d36406429504d36ab670d84f4464efa (127.15.129.129). New cstate: current_term: 1 leader_uuid: "3d36406429504d36ab670d84f4464efa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } }
W20240502 07:58:17.327666 18900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:17.327984 18900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:17.332262 18900 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
INSERT report
rows total: 2000
time total: 57.518 ms
time per row: 0.028759 ms
I20240502 07:58:17.433775 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.131:43267
--local_ip_for_outbound_sockets=127.15.129.131
--tserver_master_addrs=127.15.129.190:41083
--webserver_port=40647
--webserver_interface=127.15.129.131
--builtin_ntp_servers=127.15.129.148:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 07:58:17.541131 18930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:17.541402 18930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:17.541482 18930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:17.544473 18930 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:17.544549 18930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:17.544651 18930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.131
I20240502 07:58:17.548261 18930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.131:43267
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.15.129.131
--webserver_port=40647
--tserver_master_addrs=127.15.129.190:41083
--never_fsync=true
--heap_profile_path=/tmp/kudu.18930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.131
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:17.549176 18930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:17.550299 18930 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240502 07:58:17.559864 18930 server_base.cc:1034] running on GCE node
W20240502 07:58:17.561183 18935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:17.562134 18938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:17.563216 18936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:17.563676 18930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:17.564270 18930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:17.565383 18930 hybrid_clock.cc:648] HybridClock initialized: now 1714636697565366 us; error 44 us; skew 500 ppm
I20240502 07:58:17.567304 18930 webserver.cc:460] Webserver started at http://127.15.129.131:40647/ using document root <none> and password file <none>
I20240502 07:58:17.567853 18930 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:17.567932 18930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:17.571396 18930 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:58:17.573206 18944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:17.574198 18930 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:17.574302 18930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f"
format_stamp: "Formatted at 2024-05-02 07:58:11 on dist-test-slave-0hx2"
I20240502 07:58:17.574685 18930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:17.600152 18930 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:17.600798 18930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:17.600960 18930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:17.601533 18930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:17.602406 18930 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:17.602455 18930 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:17.602516 18930 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:17.602547 18930 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:17.611196 18930 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.131:43267
I20240502 07:58:17.611232 19057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.131:43267 every 8 connection(s)
I20240502 07:58:17.612241 18930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 07:58:17.619331 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 18930
I20240502 07:58:17.619793 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.132:43973
--local_ip_for_outbound_sockets=127.15.129.132
--tserver_master_addrs=127.15.129.190:41083
--webserver_port=41309
--webserver_interface=127.15.129.132
--builtin_ntp_servers=127.15.129.148:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:17.622288 19058 heartbeater.cc:347] Connected to a master server at 127.15.129.190:41083
I20240502 07:58:17.622509 19058 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:17.622954 19058 heartbeater.cc:511] Master 127.15.129.190:41083 requested a full tablet report, sending...
I20240502 07:58:17.624032 17732 ts_manager.cc:194] Re-registered known tserver with Master: 9fdc864a7d9349808a6fc7b2cf4eba8f (127.15.129.131:43267)
I20240502 07:58:17.624569 17732 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.131:60461
W20240502 07:58:17.727669 19061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:17.727906 19061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:17.727954 19061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:17.731091 19061 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:17.731165 19061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:17.731256 19061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.132
I20240502 07:58:17.735059 19061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.132:43973
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.15.129.132
--webserver_port=41309
--tserver_master_addrs=127.15.129.190:41083
--never_fsync=true
--heap_profile_path=/tmp/kudu.19061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.132
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:17.735988 19061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:17.737111 19061 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:17.746037 19066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:17.746903 19067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:17.748245 19069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:17.749509 19061 server_base.cc:1034] running on GCE node
I20240502 07:58:17.749890 19061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:17.750459 19061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:17.751607 19061 hybrid_clock.cc:648] HybridClock initialized: now 1714636697751597 us; error 30 us; skew 500 ppm
I20240502 07:58:17.753692 19061 webserver.cc:460] Webserver started at http://127.15.129.132:41309/ using document root <none> and password file <none>
I20240502 07:58:17.754269 19061 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:17.754348 19061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:17.757905 19061 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:58:17.759863 19075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:17.760883 19061 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:17.760993 19061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "3f7795cc262b43cfa8a9252c424b7354"
format_stamp: "Formatted at 2024-05-02 07:58:11 on dist-test-slave-0hx2"
I20240502 07:58:17.761461 19061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:17.810647 19061 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:17.811295 19061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:17.811452 19061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:17.811975 19061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:17.812827 19061 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:17.812875 19061 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:17.812929 19061 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:17.812971 19061 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:17.821434 19061 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.132:43973
I20240502 07:58:17.821484 19188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.132:43973 every 8 connection(s)
I20240502 07:58:17.822427 19061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 07:58:17.826205 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 19061
I20240502 07:58:17.826575 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.133:34385
--local_ip_for_outbound_sockets=127.15.129.133
--tserver_master_addrs=127.15.129.190:41083
--webserver_port=37597
--webserver_interface=127.15.129.133
--builtin_ntp_servers=127.15.129.148:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:17.831025 19189 heartbeater.cc:347] Connected to a master server at 127.15.129.190:41083
I20240502 07:58:17.831233 19189 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:17.831642 19189 heartbeater.cc:511] Master 127.15.129.190:41083 requested a full tablet report, sending...
I20240502 07:58:17.832556 17732 ts_manager.cc:194] Re-registered known tserver with Master: 3f7795cc262b43cfa8a9252c424b7354 (127.15.129.132:43973)
I20240502 07:58:17.833145 17732 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.132:38993
W20240502 07:58:17.935654 19192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:17.935882 19192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:17.935953 19192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:17.939136 19192 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:17.939219 19192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:17.939307 19192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.133
I20240502 07:58:17.943183 19192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.133:34385
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.15.129.133
--webserver_port=37597
--tserver_master_addrs=127.15.129.190:41083
--never_fsync=true
--heap_profile_path=/tmp/kudu.19192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.133
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:17.944061 19192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:17.945180 19192 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:17.954066 19197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:17.954990 19200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:17.956331 19198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:17.957587 19192 server_base.cc:1034] running on GCE node
I20240502 07:58:17.957973 19192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:17.958503 19192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:17.959658 19192 hybrid_clock.cc:648] HybridClock initialized: now 1714636697959649 us; error 24 us; skew 500 ppm
I20240502 07:58:17.961534 19192 webserver.cc:460] Webserver started at http://127.15.129.133:37597/ using document root <none> and password file <none>
I20240502 07:58:17.962082 19192 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:17.962158 19192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:17.965540 19192 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20240502 07:58:17.967496 19206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:17.968514 19192 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:17.968624 19192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "451db10510a9469b92abc0705d7bf44d"
format_stamp: "Formatted at 2024-05-02 07:58:11 on dist-test-slave-0hx2"
I20240502 07:58:17.969033 19192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:17.980330 19192 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:17.980902 19192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:17.981056 19192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:17.981582 19192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:17.982615 19213 ts_tablet_manager.cc:516] Loading tablet metadata (0/1 complete)
I20240502 07:58:17.985618 19192 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240502 07:58:17.985672 19192 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20240502 07:58:17.985747 19192 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240502 07:58:17.987051 19192 ts_tablet_manager.cc:590] Registered 1 tablets
I20240502 07:58:17.987092 19192 ts_tablet_manager.cc:569] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:17.987326 19213 tablet_bootstrap.cc:492] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d: Bootstrap starting.
I20240502 07:58:17.997152 19192 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.133:34385
I20240502 07:58:17.997201 19320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.133:34385 every 8 connection(s)
I20240502 07:58:17.998175 19192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 07:58:18.000795 19213 log.cc:824] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:18.002112 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 19192
I20240502 07:58:18.002514 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.134:34275
--local_ip_for_outbound_sockets=127.15.129.134
--tserver_master_addrs=127.15.129.190:41083
--webserver_port=33535
--webserver_interface=127.15.129.134
--builtin_ntp_servers=127.15.129.148:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:18.003412 19213 tablet_bootstrap.cc:492] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:18.003767 19213 tablet_bootstrap.cc:492] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d: Bootstrap complete.
I20240502 07:58:18.004994 19213 ts_tablet_manager.cc:1367] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.005s
I20240502 07:58:18.006305 19321 heartbeater.cc:347] Connected to a master server at 127.15.129.190:41083
I20240502 07:58:18.006510 19321 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:18.006944 19321 heartbeater.cc:511] Master 127.15.129.190:41083 requested a full tablet report, sending...
I20240502 07:58:18.007871 19213 raft_consensus.cc:348] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } }
I20240502 07:58:18.008280 19213 raft_consensus.cc:729] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 451db10510a9469b92abc0705d7bf44d, State: Initialized, Role: FOLLOWER
I20240502 07:58:18.008603 17732 ts_manager.cc:194] Re-registered known tserver with Master: 451db10510a9469b92abc0705d7bf44d (127.15.129.133:34385)
I20240502 07:58:18.009510 17732 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.133:36401
I20240502 07:58:18.009562 19213 consensus_queue.cc:260] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } }
I20240502 07:58:18.009804 19213 raft_consensus.cc:388] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:18.009898 19213 raft_consensus.cc:482] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:18.010054 19213 raft_consensus.cc:3037] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:18.012446 19213 raft_consensus.cc:504] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } }
I20240502 07:58:18.012835 19213 leader_election.cc:302] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 451db10510a9469b92abc0705d7bf44d; no voters:
I20240502 07:58:18.013202 19213 leader_election.cc:288] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240502 07:58:18.013307 19326 raft_consensus.cc:2781] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:58:18.013526 19326 raft_consensus.cc:686] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [term 2 LEADER]: Becoming Leader. State: Replica: 451db10510a9469b92abc0705d7bf44d, State: Running, Role: LEADER
I20240502 07:58:18.013814 19321 heartbeater.cc:503] Master 127.15.129.190:41083 was elected leader, sending a full tablet report...
I20240502 07:58:18.014194 19213 ts_tablet_manager.cc:1398] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d: Time spent starting tablet: real 0.009s user 0.008s sys 0.001s
I20240502 07:58:18.014174 19326 consensus_queue.cc:237] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } }
I20240502 07:58:18.016387 17732 catalog_manager.cc:5484] T 41a4e8bf2f494f159b25a8437f7bdeec P 451db10510a9469b92abc0705d7bf44d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "451db10510a9469b92abc0705d7bf44d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } } }
W20240502 07:58:18.114645 19325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:18.114853 19325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:18.114908 19325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:18.117987 19325 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:18.118049 19325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:18.118124 19325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.134
I20240502 07:58:18.121704 19325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.134:34275
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.15.129.134
--webserver_port=33535
--tserver_master_addrs=127.15.129.190:41083
--never_fsync=true
--heap_profile_path=/tmp/kudu.19325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.134
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:18.122612 19325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:18.123915 19325 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:18.133293 19338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:18.134179 19339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:18.135485 19341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:18.136744 19325 server_base.cc:1034] running on GCE node
I20240502 07:58:18.137123 19325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:18.137701 19325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:18.138851 19325 hybrid_clock.cc:648] HybridClock initialized: now 1714636698138833 us; error 37 us; skew 500 ppm
I20240502 07:58:18.140623 19325 webserver.cc:460] Webserver started at http://127.15.129.134:33535/ using document root <none> and password file <none>
I20240502 07:58:18.141170 19325 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:18.141247 19325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:18.144562 19325 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:18.146461 19347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:18.147457 19325 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:18.147570 19325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "c815aa98415f4682b2c8429a67e85444"
format_stamp: "Formatted at 2024-05-02 07:58:12 on dist-test-slave-0hx2"
I20240502 07:58:18.147984 19325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:18.159847 19325 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:18.160482 19325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:18.160641 19325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:18.161165 19325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:18.162115 19325 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:18.162163 19325 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:18.162225 19325 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:18.162254 19325 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:18.170877 19325 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.134:34275
I20240502 07:58:18.170897 19460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.134:34275 every 8 connection(s)
I20240502 07:58:18.171923 19325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 07:58:18.172827 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 19325
I20240502 07:58:18.173224 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.135:37247
--local_ip_for_outbound_sockets=127.15.129.135
--tserver_master_addrs=127.15.129.190:41083
--webserver_port=38639
--webserver_interface=127.15.129.135
--builtin_ntp_servers=127.15.129.148:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:18.182197 19461 heartbeater.cc:347] Connected to a master server at 127.15.129.190:41083
I20240502 07:58:18.182461 19461 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:18.183048 19461 heartbeater.cc:511] Master 127.15.129.190:41083 requested a full tablet report, sending...
I20240502 07:58:18.184144 17732 ts_manager.cc:194] Re-registered known tserver with Master: c815aa98415f4682b2c8429a67e85444 (127.15.129.134:34275)
I20240502 07:58:18.184733 17732 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.134:34301
W20240502 07:58:18.282712 19464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:18.282935 19464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:18.282980 19464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:18.286125 19464 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:18.286190 19464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:18.286288 19464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.135
I20240502 07:58:18.289997 19464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.135:37247
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.15.129.135
--webserver_port=38639
--tserver_master_addrs=127.15.129.190:41083
--never_fsync=true
--heap_profile_path=/tmp/kudu.19464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.135
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:18.290865 19464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:18.291966 19464 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:18.301458 19472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:18.302671 19469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:18.303732 19470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:18.304857 19464 server_base.cc:1034] running on GCE node
I20240502 07:58:18.305275 19464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:18.305888 19464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:18.307061 19464 hybrid_clock.cc:648] HybridClock initialized: now 1714636698307050 us; error 33 us; skew 500 ppm
I20240502 07:58:18.309265 19464 webserver.cc:460] Webserver started at http://127.15.129.135:38639/ using document root <none> and password file <none>
I20240502 07:58:18.309870 19464 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:18.309957 19464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:18.313393 19464 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.002s
I20240502 07:58:18.315413 19478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:18.316402 19464 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:18.316514 19464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "d87eea2501954fca83327b9408429b66"
format_stamp: "Formatted at 2024-05-02 07:58:12 on dist-test-slave-0hx2"
I20240502 07:58:18.316944 19464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:18.331413 19464 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:18.332157 19464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:18.332315 19464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:18.332906 19464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:18.333963 19464 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:18.334018 19464 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:18.334067 19464 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:18.334136 19464 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:18.342736 19464 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.135:37247
I20240502 07:58:18.342757 19591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.135:37247 every 8 connection(s)
I20240502 07:58:18.343871 19464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 07:58:18.349584 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 19464
I20240502 07:58:18.357932 19592 heartbeater.cc:347] Connected to a master server at 127.15.129.190:41083
I20240502 07:58:18.358168 19592 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:18.358624 19592 heartbeater.cc:511] Master 127.15.129.190:41083 requested a full tablet report, sending...
I20240502 07:58:18.359647 17732 ts_manager.cc:194] Re-registered known tserver with Master: d87eea2501954fca83327b9408429b66 (127.15.129.135:37247)
I20240502 07:58:18.360430 17732 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.135:59481
I20240502 07:58:18.541735 17853 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:58:18.542315 18992 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:18.550086 19255 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:18.550690 19395 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:18.551419 19526 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:18.551782 19123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:18.560495 17984 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:58:18.626436 19058 heartbeater.cc:503] Master 127.15.129.190:41083 was elected leader, sending a full tablet report...
I20240502 07:58:18.672636 19255 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:18.674130 17984 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:58:18.676916 19123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:18.682216 17853 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:58:18.696079 18992 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:18.702420 19395 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:18.703202 19526 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:18.749868 17874 consensus_queue.cc:237] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } }
W20240502 07:58:18.755182 17810 consensus_peers.cc:485] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa -> Peer 3f7795cc262b43cfa8a9252c424b7354 (127.15.129.132:43973): Couldn't send request to peer 3f7795cc262b43cfa8a9252c424b7354. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 62ba9d55df884c568ea6285701011ceb. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:18.834966 19189 heartbeater.cc:503] Master 127.15.129.190:41083 was elected leader, sending a full tablet report...
I20240502 07:58:19.187029 19461 heartbeater.cc:503] Master 127.15.129.190:41083 was elected leader, sending a full tablet report...
I20240502 07:58:19.346333 19695 ts_tablet_manager.cc:902] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354: Initiating tablet copy from peer 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343)
I20240502 07:58:19.347081 19695 tablet_copy_client.cc:308] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.129:42343
I20240502 07:58:19.351554 17894 tablet_copy_service.cc:140] P 3d36406429504d36ab670d84f4464efa: Received BeginTabletCopySession request for tablet 62ba9d55df884c568ea6285701011ceb from peer 3f7795cc262b43cfa8a9252c424b7354 ({username='slave'} at 127.15.129.132:56575)
I20240502 07:58:19.351716 17894 tablet_copy_service.cc:161] P 3d36406429504d36ab670d84f4464efa: Beginning new tablet copy session on tablet 62ba9d55df884c568ea6285701011ceb from peer 3f7795cc262b43cfa8a9252c424b7354 at {username='slave'} at 127.15.129.132:56575: session id = 3f7795cc262b43cfa8a9252c424b7354-62ba9d55df884c568ea6285701011ceb
I20240502 07:58:19.353430 17894 tablet_copy_source_session.cc:205] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:19.356683 19695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62ba9d55df884c568ea6285701011ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:19.361829 19695 tablet_copy_client.cc:783] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:19.362001 19695 tablet_copy_client.cc:647] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:19.362361 19592 heartbeater.cc:503] Master 127.15.129.190:41083 was elected leader, sending a full tablet report...
I20240502 07:58:19.363916 19695 tablet_copy_client.cc:523] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:19.365475 19695 tablet_bootstrap.cc:492] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354: Bootstrap starting.
I20240502 07:58:19.370182 19695 log.cc:824] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:19.388525 19695 tablet_bootstrap.cc:492] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 07:58:19.388911 19695 tablet_bootstrap.cc:492] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354: Bootstrap complete.
I20240502 07:58:19.389168 19695 ts_tablet_manager.cc:1367] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354: Time spent bootstrapping tablet: real 0.024s user 0.021s sys 0.000s
I20240502 07:58:19.391122 19695 raft_consensus.cc:348] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:19.391645 19695 raft_consensus.cc:729] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f7795cc262b43cfa8a9252c424b7354, State: Initialized, Role: FOLLOWER
I20240502 07:58:19.391927 19695 consensus_queue.cc:260] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } }
I20240502 07:58:19.392735 19695 ts_tablet_manager.cc:1398] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
I20240502 07:58:19.393622 17894 tablet_copy_service.cc:342] P 3d36406429504d36ab670d84f4464efa: Request end of tablet copy session 3f7795cc262b43cfa8a9252c424b7354-62ba9d55df884c568ea6285701011ceb received from {username='slave'} at 127.15.129.132:56575
I20240502 07:58:19.393713 17894 tablet_copy_service.cc:433] P 3d36406429504d36ab670d84f4464efa: ending tablet copy session 3f7795cc262b43cfa8a9252c424b7354-62ba9d55df884c568ea6285701011ceb on tablet 62ba9d55df884c568ea6285701011ceb with peer 3f7795cc262b43cfa8a9252c424b7354
I20240502 07:58:19.960440 19143 raft_consensus.cc:1203] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:19.961879 19706 raft_consensus.cc:2932] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3f7795cc262b43cfa8a9252c424b7354 (127.15.129.132) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } } }
I20240502 07:58:19.962500 19143 raft_consensus.cc:2932] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3f7795cc262b43cfa8a9252c424b7354 (127.15.129.132) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } } }
I20240502 07:58:19.963564 19706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:50690 (request call id 0) took 1214 ms. Trace:
I20240502 07:58:19.963647 19706 rpcz_store.cc:276] 0502 07:58:18.749164 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 07:58:18.749229 (+ 65us) service_pool.cc:225] Handling call
0502 07:58:18.750664 (+ 1435us) log.cc:842] Serialized 263 byte log entry
0502 07:58:19.963544 (+1212880us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":48,"raft.queue_time_us":87,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":323,"thread_start_us":119,"threads_started":2,"wal-append.queue_time_us":172,"wal-append.run_cpu_time_us":375,"wal-append.run_wall_time_us":1000462}
I20240502 07:58:19.965299 17732 catalog_manager.cc:5484] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 reported cstate change: config changed from index -1 to 4, VOTER 3f7795cc262b43cfa8a9252c424b7354 (127.15.129.132) added. New cstate: current_term: 1 leader_uuid: "3d36406429504d36ab670d84f4464efa" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } } }
I20240502 07:58:19.984592 18005 consensus_queue.cc:237] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } }
W20240502 07:58:19.989902 17941 consensus_peers.cc:485] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 -> Peer 9fdc864a7d9349808a6fc7b2cf4eba8f (127.15.129.131:43267): Couldn't send request to peer 9fdc864a7d9349808a6fc7b2cf4eba8f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e82159be5de549fe9ed93db5ef6262fc. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:20.422325 19734 ts_tablet_manager.cc:902] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f: Initiating tablet copy from peer 923d7729977f4edda0aa760e184a8830 (127.15.129.130:45189)
I20240502 07:58:20.423048 19734 tablet_copy_client.cc:308] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.130:45189
I20240502 07:58:20.426954 18025 tablet_copy_service.cc:140] P 923d7729977f4edda0aa760e184a8830: Received BeginTabletCopySession request for tablet e82159be5de549fe9ed93db5ef6262fc from peer 9fdc864a7d9349808a6fc7b2cf4eba8f ({username='slave'} at 127.15.129.131:56885)
I20240502 07:58:20.427095 18025 tablet_copy_service.cc:161] P 923d7729977f4edda0aa760e184a8830: Beginning new tablet copy session on tablet e82159be5de549fe9ed93db5ef6262fc from peer 9fdc864a7d9349808a6fc7b2cf4eba8f at {username='slave'} at 127.15.129.131:56885: session id = 9fdc864a7d9349808a6fc7b2cf4eba8f-e82159be5de549fe9ed93db5ef6262fc
I20240502 07:58:20.428215 18025 tablet_copy_source_session.cc:205] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:20.431308 19734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e82159be5de549fe9ed93db5ef6262fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:20.436861 19734 tablet_copy_client.cc:783] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:20.437088 19734 tablet_copy_client.cc:647] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:20.438915 19734 tablet_copy_client.cc:523] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:20.440521 19734 tablet_bootstrap.cc:492] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f: Bootstrap starting.
I20240502 07:58:20.445935 19734 log.cc:824] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:20.464350 19734 tablet_bootstrap.cc:492] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 07:58:20.464743 19734 tablet_bootstrap.cc:492] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f: Bootstrap complete.
I20240502 07:58:20.465029 19734 ts_tablet_manager.cc:1367] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f: Time spent bootstrapping tablet: real 0.025s user 0.022s sys 0.000s
I20240502 07:58:20.467133 19734 raft_consensus.cc:348] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:20.467705 19734 raft_consensus.cc:729] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fdc864a7d9349808a6fc7b2cf4eba8f, State: Initialized, Role: FOLLOWER
I20240502 07:58:20.468027 19734 consensus_queue.cc:260] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } }
I20240502 07:58:20.468838 19734 ts_tablet_manager.cc:1398] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
I20240502 07:58:20.469617 18025 tablet_copy_service.cc:342] P 923d7729977f4edda0aa760e184a8830: Request end of tablet copy session 9fdc864a7d9349808a6fc7b2cf4eba8f-e82159be5de549fe9ed93db5ef6262fc received from {username='slave'} at 127.15.129.131:56885
I20240502 07:58:20.469707 18025 tablet_copy_service.cc:433] P 923d7729977f4edda0aa760e184a8830: ending tablet copy session 9fdc864a7d9349808a6fc7b2cf4eba8f-e82159be5de549fe9ed93db5ef6262fc on tablet e82159be5de549fe9ed93db5ef6262fc with peer 9fdc864a7d9349808a6fc7b2cf4eba8f
I20240502 07:58:21.034235 19012 raft_consensus.cc:1203] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:21.035475 19745 raft_consensus.cc:2932] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 9fdc864a7d9349808a6fc7b2cf4eba8f (127.15.129.131) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } } }
I20240502 07:58:21.036000 19012 raft_consensus.cc:2932] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 9fdc864a7d9349808a6fc7b2cf4eba8f (127.15.129.131) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } } }
I20240502 07:58:21.036984 19745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:45666 (request call id 0) took 1053 ms. Trace:
I20240502 07:58:21.037070 19745 rpcz_store.cc:276] 0502 07:58:19.983974 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 07:58:19.984025 (+ 51us) service_pool.cc:225] Handling call
0502 07:58:19.985456 (+ 1431us) log.cc:842] Serialized 263 byte log entry
0502 07:58:21.036966 (+1051510us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":51,"raft.queue_time_us":97,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":300,"thread_start_us":114,"threads_started":2,"wal-append.queue_time_us":152,"wal-append.run_cpu_time_us":350,"wal-append.run_wall_time_us":1000415}
I20240502 07:58:21.038316 17727 catalog_manager.cc:5484] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f reported cstate change: config changed from index -1 to 4, VOTER 9fdc864a7d9349808a6fc7b2cf4eba8f (127.15.129.131) added. New cstate: current_term: 1 leader_uuid: "923d7729977f4edda0aa760e184a8830" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } } }
I20240502 07:58:21.056815 17874 consensus_queue.cc:237] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } }
W20240502 07:58:21.061389 17809 consensus_peers.cc:485] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa -> Peer d87eea2501954fca83327b9408429b66 (127.15.129.135:37247): Couldn't send request to peer d87eea2501954fca83327b9408429b66. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bab58d3c92774b1ab9fa77e100db9073. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:21.456230 19772 ts_tablet_manager.cc:902] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66: Initiating tablet copy from peer 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343)
I20240502 07:58:21.457016 19772 tablet_copy_client.cc:308] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.129:42343
I20240502 07:58:21.460667 17894 tablet_copy_service.cc:140] P 3d36406429504d36ab670d84f4464efa: Received BeginTabletCopySession request for tablet bab58d3c92774b1ab9fa77e100db9073 from peer d87eea2501954fca83327b9408429b66 ({username='slave'} at 127.15.129.135:56657)
I20240502 07:58:21.460783 17894 tablet_copy_service.cc:161] P 3d36406429504d36ab670d84f4464efa: Beginning new tablet copy session on tablet bab58d3c92774b1ab9fa77e100db9073 from peer d87eea2501954fca83327b9408429b66 at {username='slave'} at 127.15.129.135:56657: session id = d87eea2501954fca83327b9408429b66-bab58d3c92774b1ab9fa77e100db9073
I20240502 07:58:21.461613 17894 tablet_copy_source_session.cc:205] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:21.464493 19772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bab58d3c92774b1ab9fa77e100db9073. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:21.469579 19772 tablet_copy_client.cc:783] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:21.469833 19772 tablet_copy_client.cc:647] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:21.471365 19772 tablet_copy_client.cc:523] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:21.472952 19772 tablet_bootstrap.cc:492] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66: Bootstrap starting.
I20240502 07:58:21.477806 19772 log.cc:824] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:21.497197 19772 tablet_bootstrap.cc:492] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 07:58:21.497584 19772 tablet_bootstrap.cc:492] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66: Bootstrap complete.
I20240502 07:58:21.497856 19772 ts_tablet_manager.cc:1367] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66: Time spent bootstrapping tablet: real 0.025s user 0.026s sys 0.000s
I20240502 07:58:21.499855 19772 raft_consensus.cc:348] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:21.500344 19772 raft_consensus.cc:729] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d87eea2501954fca83327b9408429b66, State: Initialized, Role: FOLLOWER
I20240502 07:58:21.500591 19772 consensus_queue.cc:260] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } }
I20240502 07:58:21.501415 19772 ts_tablet_manager.cc:1398] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240502 07:58:21.502146 17894 tablet_copy_service.cc:342] P 3d36406429504d36ab670d84f4464efa: Request end of tablet copy session d87eea2501954fca83327b9408429b66-bab58d3c92774b1ab9fa77e100db9073 received from {username='slave'} at 127.15.129.135:56657
I20240502 07:58:21.502225 17894 tablet_copy_service.cc:433] P 3d36406429504d36ab670d84f4464efa: ending tablet copy session d87eea2501954fca83327b9408429b66-bab58d3c92774b1ab9fa77e100db9073 on tablet bab58d3c92774b1ab9fa77e100db9073 with peer d87eea2501954fca83327b9408429b66
I20240502 07:58:21.850910 19546 raft_consensus.cc:1203] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:21.852056 19781 raft_consensus.cc:2932] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER d87eea2501954fca83327b9408429b66 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } } }
I20240502 07:58:21.852622 19546 raft_consensus.cc:2932] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER d87eea2501954fca83327b9408429b66 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } } }
I20240502 07:58:21.855840 17732 catalog_manager.cc:5484] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa reported cstate change: config changed from index -1 to 4, VOTER d87eea2501954fca83327b9408429b66 (127.15.129.135) added. New cstate: current_term: 1 leader_uuid: "3d36406429504d36ab670d84f4464efa" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } } }
I20240502 07:58:21.872655 18005 consensus_queue.cc:237] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } }
W20240502 07:58:21.877283 17941 consensus_peers.cc:485] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 -> Peer c815aa98415f4682b2c8429a67e85444 (127.15.129.134:34275): Couldn't send request to peer c815aa98415f4682b2c8429a67e85444. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4ae9214f8e5a4113b339c78cf80b1ed8. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:22.434731 19806 ts_tablet_manager.cc:902] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444: Initiating tablet copy from peer 923d7729977f4edda0aa760e184a8830 (127.15.129.130:45189)
I20240502 07:58:22.435411 19806 tablet_copy_client.cc:308] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.130:45189
I20240502 07:58:22.438788 18025 tablet_copy_service.cc:140] P 923d7729977f4edda0aa760e184a8830: Received BeginTabletCopySession request for tablet 4ae9214f8e5a4113b339c78cf80b1ed8 from peer c815aa98415f4682b2c8429a67e85444 ({username='slave'} at 127.15.129.134:41603)
I20240502 07:58:22.438882 18025 tablet_copy_service.cc:161] P 923d7729977f4edda0aa760e184a8830: Beginning new tablet copy session on tablet 4ae9214f8e5a4113b339c78cf80b1ed8 from peer c815aa98415f4682b2c8429a67e85444 at {username='slave'} at 127.15.129.134:41603: session id = c815aa98415f4682b2c8429a67e85444-4ae9214f8e5a4113b339c78cf80b1ed8
I20240502 07:58:22.439769 18025 tablet_copy_source_session.cc:205] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:22.442654 19806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ae9214f8e5a4113b339c78cf80b1ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:22.447502 19806 tablet_copy_client.cc:783] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:22.447698 19806 tablet_copy_client.cc:647] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:22.449048 19806 tablet_copy_client.cc:523] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:22.450433 19806 tablet_bootstrap.cc:492] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444: Bootstrap starting.
I20240502 07:58:22.455309 19806 log.cc:824] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:22.473897 19806 tablet_bootstrap.cc:492] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 07:58:22.474304 19806 tablet_bootstrap.cc:492] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444: Bootstrap complete.
I20240502 07:58:22.474597 19806 ts_tablet_manager.cc:1367] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444: Time spent bootstrapping tablet: real 0.024s user 0.020s sys 0.004s
I20240502 07:58:22.476955 19806 raft_consensus.cc:348] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:22.477563 19806 raft_consensus.cc:729] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c815aa98415f4682b2c8429a67e85444, State: Initialized, Role: FOLLOWER
I20240502 07:58:22.477845 19806 consensus_queue.cc:260] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } }
I20240502 07:58:22.478682 19806 ts_tablet_manager.cc:1398] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20240502 07:58:22.479411 18025 tablet_copy_service.cc:342] P 923d7729977f4edda0aa760e184a8830: Request end of tablet copy session c815aa98415f4682b2c8429a67e85444-4ae9214f8e5a4113b339c78cf80b1ed8 received from {username='slave'} at 127.15.129.134:41603
I20240502 07:58:22.479488 18025 tablet_copy_service.cc:433] P 923d7729977f4edda0aa760e184a8830: ending tablet copy session c815aa98415f4682b2c8429a67e85444-4ae9214f8e5a4113b339c78cf80b1ed8 on tablet 4ae9214f8e5a4113b339c78cf80b1ed8 with peer c815aa98415f4682b2c8429a67e85444
I20240502 07:58:23.019204 19415 raft_consensus.cc:1203] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:23.020614 19816 raft_consensus.cc:2932] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER c815aa98415f4682b2c8429a67e85444 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } } }
I20240502 07:58:23.021786 19415 raft_consensus.cc:2932] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER c815aa98415f4682b2c8429a67e85444 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } } }
I20240502 07:58:23.022111 19816 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:45688 (request call id 0) took 1149 ms. Trace:
I20240502 07:58:23.022166 19816 rpcz_store.cc:276] 0502 07:58:21.872270 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 07:58:21.872335 (+ 65us) service_pool.cc:225] Handling call
0502 07:58:21.873140 (+ 805us) log.cc:842] Serialized 263 byte log entry
0502 07:58:23.022096 (+1148956us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":64,"raft.queue_time_us":45,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":180,"thread_start_us":67,"threads_started":1,"wal-append.queue_time_us":132,"wal-append.run_cpu_time_us":352,"wal-append.run_wall_time_us":1000428}
I20240502 07:58:23.023742 17727 catalog_manager.cc:5484] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 reported cstate change: config changed from index -1 to 4, VOTER c815aa98415f4682b2c8429a67e85444 (127.15.129.134) added. New cstate: current_term: 1 leader_uuid: "923d7729977f4edda0aa760e184a8830" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } } }
I20240502 07:58:23.042886 17874 consensus_queue.cc:237] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } }
W20240502 07:58:23.047803 17808 consensus_peers.cc:485] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa -> Peer 451db10510a9469b92abc0705d7bf44d (127.15.129.133:34385): Couldn't send request to peer 451db10510a9469b92abc0705d7bf44d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f49ef9b8bef141a3bc00ec7bf7c1ac67. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:23.621382 19843 ts_tablet_manager.cc:902] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d: Initiating tablet copy from peer 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343)
I20240502 07:58:23.622066 19843 tablet_copy_client.cc:308] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.129:42343
I20240502 07:58:23.625782 17894 tablet_copy_service.cc:140] P 3d36406429504d36ab670d84f4464efa: Received BeginTabletCopySession request for tablet f49ef9b8bef141a3bc00ec7bf7c1ac67 from peer 451db10510a9469b92abc0705d7bf44d ({username='slave'} at 127.15.129.133:37093)
I20240502 07:58:23.625887 17894 tablet_copy_service.cc:161] P 3d36406429504d36ab670d84f4464efa: Beginning new tablet copy session on tablet f49ef9b8bef141a3bc00ec7bf7c1ac67 from peer 451db10510a9469b92abc0705d7bf44d at {username='slave'} at 127.15.129.133:37093: session id = 451db10510a9469b92abc0705d7bf44d-f49ef9b8bef141a3bc00ec7bf7c1ac67
I20240502 07:58:23.626701 17894 tablet_copy_source_session.cc:205] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:23.628268 19843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f49ef9b8bef141a3bc00ec7bf7c1ac67. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:23.630940 19843 tablet_copy_client.cc:783] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:23.631124 19843 tablet_copy_client.cc:647] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:23.632555 19843 tablet_copy_client.cc:523] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:23.633762 19843 tablet_bootstrap.cc:492] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d: Bootstrap starting.
I20240502 07:58:23.661087 19843 tablet_bootstrap.cc:492] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 07:58:23.661468 19843 tablet_bootstrap.cc:492] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d: Bootstrap complete.
I20240502 07:58:23.661648 19843 ts_tablet_manager.cc:1367] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d: Time spent bootstrapping tablet: real 0.028s user 0.020s sys 0.008s
I20240502 07:58:23.662165 19843 raft_consensus.cc:348] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:23.662438 19843 raft_consensus.cc:729] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 451db10510a9469b92abc0705d7bf44d, State: Initialized, Role: FOLLOWER
I20240502 07:58:23.662606 19843 consensus_queue.cc:260] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } }
I20240502 07:58:23.662976 19843 ts_tablet_manager.cc:1398] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:23.663715 17894 tablet_copy_service.cc:342] P 3d36406429504d36ab670d84f4464efa: Request end of tablet copy session 451db10510a9469b92abc0705d7bf44d-f49ef9b8bef141a3bc00ec7bf7c1ac67 received from {username='slave'} at 127.15.129.133:37093
I20240502 07:58:23.663807 17894 tablet_copy_service.cc:433] P 3d36406429504d36ab670d84f4464efa: ending tablet copy session 451db10510a9469b92abc0705d7bf44d-f49ef9b8bef141a3bc00ec7bf7c1ac67 on tablet f49ef9b8bef141a3bc00ec7bf7c1ac67 with peer 451db10510a9469b92abc0705d7bf44d
I20240502 07:58:24.133591 19275 raft_consensus.cc:1203] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:24.134732 19852 raft_consensus.cc:2932] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 451db10510a9469b92abc0705d7bf44d (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } } }
I20240502 07:58:24.135175 19275 raft_consensus.cc:2932] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 451db10510a9469b92abc0705d7bf44d (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } } }
I20240502 07:58:24.136194 19852 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:50736 (request call id 0) took 1093 ms. Trace:
I20240502 07:58:24.136303 19852 rpcz_store.cc:276] 0502 07:58:23.042482 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 07:58:23.042546 (+ 64us) service_pool.cc:225] Handling call
0502 07:58:23.043437 (+ 891us) log.cc:842] Serialized 263 byte log entry
0502 07:58:24.136178 (+1092741us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":51,"raft.queue_time_us":23,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":121,"spinlock_wait_cycles":5888,"thread_start_us":88,"threads_started":1,"wal-append.queue_time_us":170,"wal-append.run_cpu_time_us":391,"wal-append.run_wall_time_us":1000467}
I20240502 07:58:24.137790 17727 catalog_manager.cc:5484] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa reported cstate change: config changed from index -1 to 4, VOTER 451db10510a9469b92abc0705d7bf44d (127.15.129.133) added. New cstate: current_term: 1 leader_uuid: "3d36406429504d36ab670d84f4464efa" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } } }
I20240502 07:58:24.155373 17874 consensus_queue.cc:237] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } }
W20240502 07:58:24.156800 17810 consensus_peers.cc:485] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa -> Peer 3f7795cc262b43cfa8a9252c424b7354 (127.15.129.132:43973): Couldn't send request to peer 3f7795cc262b43cfa8a9252c424b7354. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 323b33d450384c90b13a1ac565f411a4. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:24.707226 19877 ts_tablet_manager.cc:902] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354: Initiating tablet copy from peer 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343)
I20240502 07:58:24.707681 19877 tablet_copy_client.cc:308] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.129:42343
I20240502 07:58:24.708228 17894 tablet_copy_service.cc:140] P 3d36406429504d36ab670d84f4464efa: Received BeginTabletCopySession request for tablet 323b33d450384c90b13a1ac565f411a4 from peer 3f7795cc262b43cfa8a9252c424b7354 ({username='slave'} at 127.15.129.132:56575)
I20240502 07:58:24.708338 17894 tablet_copy_service.cc:161] P 3d36406429504d36ab670d84f4464efa: Beginning new tablet copy session on tablet 323b33d450384c90b13a1ac565f411a4 from peer 3f7795cc262b43cfa8a9252c424b7354 at {username='slave'} at 127.15.129.132:56575: session id = 3f7795cc262b43cfa8a9252c424b7354-323b33d450384c90b13a1ac565f411a4
I20240502 07:58:24.709141 17894 tablet_copy_source_session.cc:205] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:24.709987 19877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 323b33d450384c90b13a1ac565f411a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:24.711720 19877 tablet_copy_client.cc:783] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:24.711884 19877 tablet_copy_client.cc:647] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:24.713021 19877 tablet_copy_client.cc:523] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:24.714026 19877 tablet_bootstrap.cc:492] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354: Bootstrap starting.
I20240502 07:58:24.735543 19877 tablet_bootstrap.cc:492] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 07:58:24.735819 19877 tablet_bootstrap.cc:492] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354: Bootstrap complete.
I20240502 07:58:24.735966 19877 ts_tablet_manager.cc:1367] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354: Time spent bootstrapping tablet: real 0.022s user 0.020s sys 0.000s
I20240502 07:58:24.736488 19877 raft_consensus.cc:348] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:24.736678 19877 raft_consensus.cc:729] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f7795cc262b43cfa8a9252c424b7354, State: Initialized, Role: FOLLOWER
I20240502 07:58:24.736778 19877 consensus_queue.cc:260] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } }
I20240502 07:58:24.737140 19877 ts_tablet_manager.cc:1398] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 07:58:24.737792 17894 tablet_copy_service.cc:342] P 3d36406429504d36ab670d84f4464efa: Request end of tablet copy session 3f7795cc262b43cfa8a9252c424b7354-323b33d450384c90b13a1ac565f411a4 received from {username='slave'} at 127.15.129.132:56575
I20240502 07:58:24.737882 17894 tablet_copy_service.cc:433] P 3d36406429504d36ab670d84f4464efa: ending tablet copy session 3f7795cc262b43cfa8a9252c424b7354-323b33d450384c90b13a1ac565f411a4 on tablet 323b33d450384c90b13a1ac565f411a4 with peer 3f7795cc262b43cfa8a9252c424b7354
I20240502 07:58:25.083572 19143 raft_consensus.cc:1203] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:25.084561 19884 raft_consensus.cc:2932] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3f7795cc262b43cfa8a9252c424b7354 (127.15.129.132) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } } }
I20240502 07:58:25.084930 19143 raft_consensus.cc:2932] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 3f7795cc262b43cfa8a9252c424b7354 (127.15.129.132) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } } }
I20240502 07:58:25.087684 17727 catalog_manager.cc:5484] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa reported cstate change: config changed from index -1 to 4, VOTER 3f7795cc262b43cfa8a9252c424b7354 (127.15.129.132) added. New cstate: current_term: 1 leader_uuid: "3d36406429504d36ab670d84f4464efa" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } } }
I20240502 07:58:25.104259 18005 consensus_queue.cc:237] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } }
W20240502 07:58:25.105650 17941 consensus_peers.cc:485] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 -> Peer 9fdc864a7d9349808a6fc7b2cf4eba8f (127.15.129.131:43267): Couldn't send request to peer 9fdc864a7d9349808a6fc7b2cf4eba8f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a7a2522085124c63a938200b3e925cdb. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:25.619807 19907 ts_tablet_manager.cc:902] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f: Initiating tablet copy from peer 923d7729977f4edda0aa760e184a8830 (127.15.129.130:45189)
I20240502 07:58:25.620280 19907 tablet_copy_client.cc:308] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.130:45189
I20240502 07:58:25.620858 18025 tablet_copy_service.cc:140] P 923d7729977f4edda0aa760e184a8830: Received BeginTabletCopySession request for tablet a7a2522085124c63a938200b3e925cdb from peer 9fdc864a7d9349808a6fc7b2cf4eba8f ({username='slave'} at 127.15.129.131:56885)
I20240502 07:58:25.620996 18025 tablet_copy_service.cc:161] P 923d7729977f4edda0aa760e184a8830: Beginning new tablet copy session on tablet a7a2522085124c63a938200b3e925cdb from peer 9fdc864a7d9349808a6fc7b2cf4eba8f at {username='slave'} at 127.15.129.131:56885: session id = 9fdc864a7d9349808a6fc7b2cf4eba8f-a7a2522085124c63a938200b3e925cdb
I20240502 07:58:25.622171 18025 tablet_copy_source_session.cc:205] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:25.623183 19907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7a2522085124c63a938200b3e925cdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:25.624822 19907 tablet_copy_client.cc:783] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:25.624971 19907 tablet_copy_client.cc:647] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:25.626201 19907 tablet_copy_client.cc:523] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:25.627249 19907 tablet_bootstrap.cc:492] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f: Bootstrap starting.
I20240502 07:58:25.644286 19907 tablet_bootstrap.cc:492] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 07:58:25.644552 19907 tablet_bootstrap.cc:492] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f: Bootstrap complete.
I20240502 07:58:25.644711 19907 ts_tablet_manager.cc:1367] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f: Time spent bootstrapping tablet: real 0.017s user 0.015s sys 0.000s
I20240502 07:58:25.645171 19907 raft_consensus.cc:348] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:25.645421 19907 raft_consensus.cc:729] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fdc864a7d9349808a6fc7b2cf4eba8f, State: Initialized, Role: FOLLOWER
I20240502 07:58:25.645529 19907 consensus_queue.cc:260] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } }
I20240502 07:58:25.645864 19907 ts_tablet_manager.cc:1398] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 07:58:25.646508 18025 tablet_copy_service.cc:342] P 923d7729977f4edda0aa760e184a8830: Request end of tablet copy session 9fdc864a7d9349808a6fc7b2cf4eba8f-a7a2522085124c63a938200b3e925cdb received from {username='slave'} at 127.15.129.131:56885
I20240502 07:58:25.646646 18025 tablet_copy_service.cc:433] P 923d7729977f4edda0aa760e184a8830: ending tablet copy session 9fdc864a7d9349808a6fc7b2cf4eba8f-a7a2522085124c63a938200b3e925cdb on tablet a7a2522085124c63a938200b3e925cdb with peer 9fdc864a7d9349808a6fc7b2cf4eba8f
I20240502 07:58:26.172946 19012 raft_consensus.cc:1203] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:26.173988 19915 raft_consensus.cc:2932] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 9fdc864a7d9349808a6fc7b2cf4eba8f (127.15.129.131) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } } }
I20240502 07:58:26.174464 19012 raft_consensus.cc:2932] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 9fdc864a7d9349808a6fc7b2cf4eba8f (127.15.129.131) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } } }
I20240502 07:58:26.175750 19915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:37264 (request call id 0) took 1071 ms. Trace:
I20240502 07:58:26.175866 19915 rpcz_store.cc:276] 0502 07:58:25.103931 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 07:58:25.103990 (+ 59us) service_pool.cc:225] Handling call
0502 07:58:25.104653 (+ 663us) log.cc:842] Serialized 263 byte log entry
0502 07:58:26.175735 (+1071082us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":48,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":154,"spinlock_wait_cycles":3840,"thread_start_us":66,"threads_started":1,"wal-append.queue_time_us":146,"wal-append.run_cpu_time_us":320,"wal-append.run_wall_time_us":1000389}
I20240502 07:58:26.176836 17727 catalog_manager.cc:5484] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f reported cstate change: config changed from index -1 to 4, VOTER 9fdc864a7d9349808a6fc7b2cf4eba8f (127.15.129.131) added. New cstate: current_term: 1 leader_uuid: "923d7729977f4edda0aa760e184a8830" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } } }
I20240502 07:58:26.196468 18005 consensus_queue.cc:237] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } }
W20240502 07:58:26.200860 17940 consensus_peers.cc:485] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 -> Peer d87eea2501954fca83327b9408429b66 (127.15.129.135:37247): Couldn't send request to peer d87eea2501954fca83327b9408429b66. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3f6c4761aef54c538693f1d3a4d80d10. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:26.744944 19942 ts_tablet_manager.cc:902] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66: Initiating tablet copy from peer 923d7729977f4edda0aa760e184a8830 (127.15.129.130:45189)
I20240502 07:58:26.745532 19942 tablet_copy_client.cc:308] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.130:45189
I20240502 07:58:26.749097 18025 tablet_copy_service.cc:140] P 923d7729977f4edda0aa760e184a8830: Received BeginTabletCopySession request for tablet 3f6c4761aef54c538693f1d3a4d80d10 from peer d87eea2501954fca83327b9408429b66 ({username='slave'} at 127.15.129.135:43809)
I20240502 07:58:26.749230 18025 tablet_copy_service.cc:161] P 923d7729977f4edda0aa760e184a8830: Beginning new tablet copy session on tablet 3f6c4761aef54c538693f1d3a4d80d10 from peer d87eea2501954fca83327b9408429b66 at {username='slave'} at 127.15.129.135:43809: session id = d87eea2501954fca83327b9408429b66-3f6c4761aef54c538693f1d3a4d80d10
I20240502 07:58:26.750047 18025 tablet_copy_source_session.cc:205] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:26.750993 19942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f6c4761aef54c538693f1d3a4d80d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:26.752930 19942 tablet_copy_client.cc:783] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:26.753085 19942 tablet_copy_client.cc:647] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:26.754715 19942 tablet_copy_client.cc:523] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:26.755818 19942 tablet_bootstrap.cc:492] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66: Bootstrap starting.
I20240502 07:58:26.772819 19942 tablet_bootstrap.cc:492] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 07:58:26.773130 19942 tablet_bootstrap.cc:492] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66: Bootstrap complete.
I20240502 07:58:26.773303 19942 ts_tablet_manager.cc:1367] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66: Time spent bootstrapping tablet: real 0.018s user 0.014s sys 0.000s
I20240502 07:58:26.773864 19942 raft_consensus.cc:348] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:26.774081 19942 raft_consensus.cc:729] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d87eea2501954fca83327b9408429b66, State: Initialized, Role: FOLLOWER
I20240502 07:58:26.774188 19942 consensus_queue.cc:260] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } }
I20240502 07:58:26.774523 19942 ts_tablet_manager.cc:1398] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:26.775175 18025 tablet_copy_service.cc:342] P 923d7729977f4edda0aa760e184a8830: Request end of tablet copy session d87eea2501954fca83327b9408429b66-3f6c4761aef54c538693f1d3a4d80d10 received from {username='slave'} at 127.15.129.135:43809
I20240502 07:58:26.775276 18025 tablet_copy_service.cc:433] P 923d7729977f4edda0aa760e184a8830: ending tablet copy session d87eea2501954fca83327b9408429b66-3f6c4761aef54c538693f1d3a4d80d10 on tablet 3f6c4761aef54c538693f1d3a4d80d10 with peer d87eea2501954fca83327b9408429b66
I20240502 07:58:27.338266 19546 raft_consensus.cc:1203] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:27.339216 19952 raft_consensus.cc:2932] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER d87eea2501954fca83327b9408429b66 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } } }
I20240502 07:58:27.339658 19546 raft_consensus.cc:2932] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER d87eea2501954fca83327b9408429b66 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } } }
I20240502 07:58:27.340768 19952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:37284 (request call id 0) took 1144 ms. Trace:
I20240502 07:58:27.340863 19952 rpcz_store.cc:276] 0502 07:58:26.196082 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 07:58:26.196163 (+ 81us) service_pool.cc:225] Handling call
0502 07:58:26.196963 (+ 800us) log.cc:842] Serialized 263 byte log entry
0502 07:58:27.340752 (+1143789us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":54,"raft.queue_time_us":44,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"spinlock_wait_cycles":2432,"thread_start_us":71,"threads_started":1,"wal-append.queue_time_us":158,"wal-append.run_cpu_time_us":399,"wal-append.run_wall_time_us":1000415}
I20240502 07:58:27.342284 17727 catalog_manager.cc:5484] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 reported cstate change: config changed from index -1 to 4, VOTER d87eea2501954fca83327b9408429b66 (127.15.129.135) added. New cstate: current_term: 1 leader_uuid: "923d7729977f4edda0aa760e184a8830" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } } }
I20240502 07:58:27.361218 17874 consensus_queue.cc:237] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } }
W20240502 07:58:27.365685 17810 consensus_peers.cc:485] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa -> Peer c815aa98415f4682b2c8429a67e85444 (127.15.129.134:34275): Couldn't send request to peer c815aa98415f4682b2c8429a67e85444. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4fb3087aff7d40eb960135010402371e. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:27.973055 19980 ts_tablet_manager.cc:902] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444: Initiating tablet copy from peer 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343)
I20240502 07:58:27.973598 19980 tablet_copy_client.cc:308] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.129:42343
I20240502 07:58:27.977131 17894 tablet_copy_service.cc:140] P 3d36406429504d36ab670d84f4464efa: Received BeginTabletCopySession request for tablet 4fb3087aff7d40eb960135010402371e from peer c815aa98415f4682b2c8429a67e85444 ({username='slave'} at 127.15.129.134:53577)
I20240502 07:58:27.977237 17894 tablet_copy_service.cc:161] P 3d36406429504d36ab670d84f4464efa: Beginning new tablet copy session on tablet 4fb3087aff7d40eb960135010402371e from peer c815aa98415f4682b2c8429a67e85444 at {username='slave'} at 127.15.129.134:53577: session id = c815aa98415f4682b2c8429a67e85444-4fb3087aff7d40eb960135010402371e
I20240502 07:58:27.977995 17894 tablet_copy_source_session.cc:205] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:27.978825 19980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fb3087aff7d40eb960135010402371e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:27.980593 19980 tablet_copy_client.cc:783] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:27.980749 19980 tablet_copy_client.cc:647] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:27.981876 19980 tablet_copy_client.cc:523] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:27.982856 19980 tablet_bootstrap.cc:492] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444: Bootstrap starting.
I20240502 07:58:27.999225 19980 tablet_bootstrap.cc:492] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 07:58:27.999532 19980 tablet_bootstrap.cc:492] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444: Bootstrap complete.
I20240502 07:58:27.999701 19980 ts_tablet_manager.cc:1367] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444: Time spent bootstrapping tablet: real 0.017s user 0.015s sys 0.000s
I20240502 07:58:28.000183 19980 raft_consensus.cc:348] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:28.000402 19980 raft_consensus.cc:729] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c815aa98415f4682b2c8429a67e85444, State: Initialized, Role: FOLLOWER
I20240502 07:58:28.000502 19980 consensus_queue.cc:260] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } }
I20240502 07:58:28.000829 19980 ts_tablet_manager.cc:1398] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 07:58:28.001480 17894 tablet_copy_service.cc:342] P 3d36406429504d36ab670d84f4464efa: Request end of tablet copy session c815aa98415f4682b2c8429a67e85444-4fb3087aff7d40eb960135010402371e received from {username='slave'} at 127.15.129.134:53577
I20240502 07:58:28.001572 17894 tablet_copy_service.cc:433] P 3d36406429504d36ab670d84f4464efa: ending tablet copy session c815aa98415f4682b2c8429a67e85444-4fb3087aff7d40eb960135010402371e on tablet 4fb3087aff7d40eb960135010402371e with peer c815aa98415f4682b2c8429a67e85444
I20240502 07:58:28.579042 19415 raft_consensus.cc:1203] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:28.579969 19990 raft_consensus.cc:2932] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER c815aa98415f4682b2c8429a67e85444 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } } }
I20240502 07:58:28.580394 19415 raft_consensus.cc:2932] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER c815aa98415f4682b2c8429a67e85444 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } } }
I20240502 07:58:28.582247 19990 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:46808 (request call id 0) took 1221 ms. Trace:
I20240502 07:58:28.582368 19990 rpcz_store.cc:276] 0502 07:58:27.360876 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 07:58:27.360945 (+ 69us) service_pool.cc:225] Handling call
0502 07:58:27.361721 (+ 776us) log.cc:842] Serialized 263 byte log entry
0502 07:58:28.582229 (+1220508us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":48,"raft.queue_time_us":25,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":112,"spinlock_wait_cycles":2432,"thread_start_us":69,"threads_started":1,"wal-append.queue_time_us":139,"wal-append.run_cpu_time_us":294,"wal-append.run_wall_time_us":1000369}
I20240502 07:58:28.582854 17727 catalog_manager.cc:5484] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 reported cstate change: config changed from index -1 to 4, VOTER c815aa98415f4682b2c8429a67e85444 (127.15.129.134) added. New cstate: current_term: 1 leader_uuid: "3d36406429504d36ab670d84f4464efa" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } } }
I20240502 07:58:28.648082 19255 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:28.649223 17853 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:58:28.649736 19123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:28.656301 18992 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:28.656325 19395 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:28.656759 19526 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:28.657420 17984 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:58:28.704447 17874 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4fb3087aff7d40eb960135010402371e"
dest_uuid: "3d36406429504d36ab670d84f4464efa"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:46832
I20240502 07:58:28.704627 17874 raft_consensus.cc:595] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Received request to transfer leadership
I20240502 07:58:28.759636 17853 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:58:28.760269 19255 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:28.760814 17984 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:58:28.763646 18992 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:28.776471 19123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:28.776722 19526 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:28.776795 19395 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:28.863243 17853 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:58:28.865057 19255 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:28.868862 18992 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:28.871361 17984 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:58:28.882383 19395 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:28.889752 19123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:28.891011 19526 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:28.924463 18005 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3f6c4761aef54c538693f1d3a4d80d10"
dest_uuid: "923d7729977f4edda0aa760e184a8830"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37336
I20240502 07:58:28.924674 18005 raft_consensus.cc:595] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:58:28.977118 19123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:28.979059 17853 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:58:28.980885 19255 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:28.982874 18992 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:28.983139 19395 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:28.986680 17984 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:58:28.994537 19952 raft_consensus.cc:979] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830: : Instructing follower d87eea2501954fca83327b9408429b66 to start an election
I20240502 07:58:28.994658 19952 raft_consensus.cc:1067] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Signalling peer d87eea2501954fca83327b9408429b66 to start an election
I20240502 07:58:28.995361 19546 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3f6c4761aef54c538693f1d3a4d80d10"
dest_uuid: "d87eea2501954fca83327b9408429b66"
from {username='slave'} at 127.15.129.130:47173
I20240502 07:58:28.995558 19546 raft_consensus.cc:482] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:58:28.995661 19546 raft_consensus.cc:3037] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:28.996716 19546 raft_consensus.cc:504] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } }
I20240502 07:58:28.997927 19546 leader_election.cc:288] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 [CANDIDATE]: Term 2 election: Requested vote from peers 923d7729977f4edda0aa760e184a8830 (127.15.129.130:45189)
I20240502 07:58:29.001257 18005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f6c4761aef54c538693f1d3a4d80d10" candidate_uuid: "d87eea2501954fca83327b9408429b66" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "923d7729977f4edda0aa760e184a8830"
I20240502 07:58:29.001461 18005 raft_consensus.cc:3032] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:58:29.001502 18005 raft_consensus.cc:729] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Running, Role: LEADER
I20240502 07:58:29.001664 18005 consensus_queue.cc:260] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } }
I20240502 07:58:29.001892 18005 raft_consensus.cc:3037] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:29.002938 18005 raft_consensus.cc:2454] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d87eea2501954fca83327b9408429b66 in term 2.
I20240502 07:58:29.003549 19481 leader_election.cc:302] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 923d7729977f4edda0aa760e184a8830, d87eea2501954fca83327b9408429b66; no voters:
I20240502 07:58:29.014423 20155 raft_consensus.cc:2781] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:58:29.014699 20155 raft_consensus.cc:686] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 [term 2 LEADER]: Becoming Leader. State: Replica: d87eea2501954fca83327b9408429b66, State: Running, Role: LEADER
I20240502 07:58:29.015699 20155 consensus_queue.cc:237] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } }
I20240502 07:58:29.018234 17731 catalog_manager.cc:5484] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 reported cstate change: term changed from 1 to 2, leader changed from 923d7729977f4edda0aa760e184a8830 (127.15.129.130) to d87eea2501954fca83327b9408429b66 (127.15.129.135). New cstate: current_term: 2 leader_uuid: "d87eea2501954fca83327b9408429b66" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } } }
I20240502 07:58:29.021615 19852 raft_consensus.cc:979] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa: : Instructing follower c815aa98415f4682b2c8429a67e85444 to start an election
I20240502 07:58:29.021706 19852 raft_consensus.cc:1067] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Signalling peer c815aa98415f4682b2c8429a67e85444 to start an election
I20240502 07:58:29.022702 19415 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4fb3087aff7d40eb960135010402371e"
dest_uuid: "c815aa98415f4682b2c8429a67e85444"
from {username='slave'} at 127.15.129.129:42031
I20240502 07:58:29.022894 19415 raft_consensus.cc:482] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:58:29.023000 19415 raft_consensus.cc:3037] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:29.024072 19415 raft_consensus.cc:504] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } }
I20240502 07:58:29.025074 19415 leader_election.cc:288] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 [CANDIDATE]: Term 2 election: Requested vote from peers 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343)
I20240502 07:58:29.025621 17874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fb3087aff7d40eb960135010402371e" candidate_uuid: "c815aa98415f4682b2c8429a67e85444" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d36406429504d36ab670d84f4464efa"
I20240502 07:58:29.025816 17874 raft_consensus.cc:3032] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:58:29.025877 17874 raft_consensus.cc:729] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Running, Role: LEADER
I20240502 07:58:29.026060 17874 consensus_queue.cc:260] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } }
I20240502 07:58:29.026335 17874 raft_consensus.cc:3037] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:29.027439 17874 raft_consensus.cc:2454] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c815aa98415f4682b2c8429a67e85444 in term 2.
I20240502 07:58:29.027992 19348 leader_election.cc:302] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 3d36406429504d36ab670d84f4464efa, c815aa98415f4682b2c8429a67e85444; no voters:
I20240502 07:58:29.028241 19989 raft_consensus.cc:2781] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:58:29.028463 19989 raft_consensus.cc:686] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 [term 2 LEADER]: Becoming Leader. State: Replica: c815aa98415f4682b2c8429a67e85444, State: Running, Role: LEADER
I20240502 07:58:29.028723 19526 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.028769 19989 consensus_queue.cc:237] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } }
I20240502 07:58:29.030970 17731 catalog_manager.cc:5484] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 reported cstate change: term changed from 1 to 2, leader changed from 3d36406429504d36ab670d84f4464efa (127.15.129.129) to c815aa98415f4682b2c8429a67e85444 (127.15.129.134). New cstate: current_term: 2 leader_uuid: "c815aa98415f4682b2c8429a67e85444" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } } }
I20240502 07:58:29.108101 19123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:29.109699 17853 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:58:29.110847 19255 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.114013 18992 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:29.122153 19526 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.126423 19395 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.135061 17984 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:58:29.163134 18005 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e82159be5de549fe9ed93db5ef6262fc"
dest_uuid: "923d7729977f4edda0aa760e184a8830"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37386
I20240502 07:58:29.163288 18005 raft_consensus.cc:595] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:58:29.218459 19255 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.218537 17984 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:58:29.219339 17853 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:58:29.224120 19526 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.226655 19395 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.240384 18992 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:29.250694 19123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:29.326509 19123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:29.326565 17984 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:58:29.334770 19255 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.334774 19395 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.354642 17853 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:58:29.355603 18992 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:29.362460 19526 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.386932 17874 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bab58d3c92774b1ab9fa77e100db9073"
dest_uuid: "3d36406429504d36ab670d84f4464efa"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:46916
I20240502 07:58:29.387102 17874 raft_consensus.cc:595] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Received request to transfer leadership
I20240502 07:58:29.404433 19952 raft_consensus.cc:979] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830: : Instructing follower 9fdc864a7d9349808a6fc7b2cf4eba8f to start an election
I20240502 07:58:29.404577 19952 raft_consensus.cc:1067] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Signalling peer 9fdc864a7d9349808a6fc7b2cf4eba8f to start an election
I20240502 07:58:29.405079 19012 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e82159be5de549fe9ed93db5ef6262fc"
dest_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f"
from {username='slave'} at 127.15.129.130:37077
I20240502 07:58:29.405287 19012 raft_consensus.cc:482] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:58:29.405460 19012 raft_consensus.cc:3037] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:29.406342 19012 raft_consensus.cc:504] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } }
I20240502 07:58:29.407191 19012 leader_election.cc:288] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f [CANDIDATE]: Term 2 election: Requested vote from peers 923d7729977f4edda0aa760e184a8830 (127.15.129.130:45189)
I20240502 07:58:29.407665 18005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e82159be5de549fe9ed93db5ef6262fc" candidate_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "923d7729977f4edda0aa760e184a8830"
I20240502 07:58:29.407794 18005 raft_consensus.cc:3032] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:58:29.407876 18005 raft_consensus.cc:729] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Running, Role: LEADER
I20240502 07:58:29.408049 18005 consensus_queue.cc:260] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } }
I20240502 07:58:29.408187 18005 raft_consensus.cc:3037] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:29.409127 18005 raft_consensus.cc:2454] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fdc864a7d9349808a6fc7b2cf4eba8f in term 2.
I20240502 07:58:29.409576 18947 leader_election.cc:302] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 923d7729977f4edda0aa760e184a8830, 9fdc864a7d9349808a6fc7b2cf4eba8f; no voters:
I20240502 07:58:29.409912 20285 raft_consensus.cc:2781] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:58:29.410113 20285 raft_consensus.cc:686] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 2 LEADER]: Becoming Leader. State: Replica: 9fdc864a7d9349808a6fc7b2cf4eba8f, State: Running, Role: LEADER
I20240502 07:58:29.410444 20285 consensus_queue.cc:237] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } }
I20240502 07:58:29.412847 17728 catalog_manager.cc:5484] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f reported cstate change: term changed from 1 to 2, leader changed from 923d7729977f4edda0aa760e184a8830 (127.15.129.130) to 9fdc864a7d9349808a6fc7b2cf4eba8f (127.15.129.131). New cstate: current_term: 2 leader_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } } }
I20240502 07:58:29.442742 17984 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:58:29.445399 19123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:29.445825 19526 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.446594 19395 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.446990 18992 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.447046 17853 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:58:29.480818 19255 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.491443 19852 raft_consensus.cc:979] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa: : Instructing follower d87eea2501954fca83327b9408429b66 to start an election
I20240502 07:58:29.491581 19852 raft_consensus.cc:1067] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Signalling peer d87eea2501954fca83327b9408429b66 to start an election
I20240502 07:58:29.492007 19546 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bab58d3c92774b1ab9fa77e100db9073"
dest_uuid: "d87eea2501954fca83327b9408429b66"
from {username='slave'} at 127.15.129.129:35323
I20240502 07:58:29.492137 19546 raft_consensus.cc:482] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:58:29.492204 19546 raft_consensus.cc:3037] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:29.493167 19546 raft_consensus.cc:504] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } }
I20240502 07:58:29.493553 19546 leader_election.cc:288] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66 [CANDIDATE]: Term 2 election: Requested vote from peers 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343)
I20240502 07:58:29.493942 17874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bab58d3c92774b1ab9fa77e100db9073" candidate_uuid: "d87eea2501954fca83327b9408429b66" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d36406429504d36ab670d84f4464efa"
I20240502 07:58:29.494051 17874 raft_consensus.cc:3032] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:58:29.494102 17874 raft_consensus.cc:729] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Running, Role: LEADER
I20240502 07:58:29.494267 17874 consensus_queue.cc:260] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } }
I20240502 07:58:29.494397 17874 raft_consensus.cc:3037] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:29.495199 17874 raft_consensus.cc:2454] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d87eea2501954fca83327b9408429b66 in term 2.
I20240502 07:58:29.495507 19479 leader_election.cc:302] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 3d36406429504d36ab670d84f4464efa, d87eea2501954fca83327b9408429b66; no voters:
I20240502 07:58:29.495666 20156 raft_consensus.cc:2781] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:58:29.495754 20156 raft_consensus.cc:686] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66 [term 2 LEADER]: Becoming Leader. State: Replica: d87eea2501954fca83327b9408429b66, State: Running, Role: LEADER
I20240502 07:58:29.495887 20156 consensus_queue.cc:237] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } }
I20240502 07:58:29.497529 17729 catalog_manager.cc:5484] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66 reported cstate change: term changed from 1 to 2, leader changed from 3d36406429504d36ab670d84f4464efa (127.15.129.129) to d87eea2501954fca83327b9408429b66 (127.15.129.135). New cstate: current_term: 2 leader_uuid: "d87eea2501954fca83327b9408429b66" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } } }
I20240502 07:58:29.510852 18005 raft_consensus.cc:1261] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Refusing update from remote peer d87eea2501954fca83327b9408429b66: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:58:29.511430 20156 consensus_queue.cc:1035] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:29.567281 19255 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.568703 19123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:29.570386 17853 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:58:29.574106 19395 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.574872 18992 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.582670 17984 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:58:29.584766 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:29.605948 17874 raft_consensus.cc:1261] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Refusing update from remote peer c815aa98415f4682b2c8429a67e85444: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:58:29.606734 20361 consensus_queue.cc:1035] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:29.622470 17874 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f49ef9b8bef141a3bc00ec7bf7c1ac67"
dest_uuid: "3d36406429504d36ab670d84f4464efa"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:46958
I20240502 07:58:29.622619 17874 raft_consensus.cc:595] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Received request to transfer leadership
I20240502 07:58:29.674193 17853 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:58:29.674609 19123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:29.677281 19255 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.687235 18992 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.688828 19395 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.690007 17984 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:58:29.699052 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:29.779184 17853 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:58:29.779609 19255 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.781518 19123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:29.785254 17984 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:58:29.786162 18992 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.795874 19395 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.797708 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:29.833745 17874 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "62ba9d55df884c568ea6285701011ceb"
dest_uuid: "3d36406429504d36ab670d84f4464efa"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:46988
I20240502 07:58:29.833885 17874 raft_consensus.cc:595] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Received request to transfer leadership
I20240502 07:58:29.874013 19852 raft_consensus.cc:979] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa: : Instructing follower 451db10510a9469b92abc0705d7bf44d to start an election
I20240502 07:58:29.874127 19852 raft_consensus.cc:1067] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Signalling peer 451db10510a9469b92abc0705d7bf44d to start an election
I20240502 07:58:29.874604 19275 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f49ef9b8bef141a3bc00ec7bf7c1ac67"
dest_uuid: "451db10510a9469b92abc0705d7bf44d"
from {username='slave'} at 127.15.129.129:56837
I20240502 07:58:29.874799 19275 raft_consensus.cc:482] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:58:29.874859 19275 raft_consensus.cc:3037] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:29.875907 19275 raft_consensus.cc:504] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } }
I20240502 07:58:29.876561 19275 leader_election.cc:288] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d [CANDIDATE]: Term 2 election: Requested vote from peers 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343)
I20240502 07:58:29.876955 17874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f49ef9b8bef141a3bc00ec7bf7c1ac67" candidate_uuid: "451db10510a9469b92abc0705d7bf44d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d36406429504d36ab670d84f4464efa"
I20240502 07:58:29.877089 17874 raft_consensus.cc:3032] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:58:29.877137 17874 raft_consensus.cc:729] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Running, Role: LEADER
I20240502 07:58:29.877357 17874 consensus_queue.cc:260] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } }
I20240502 07:58:29.877526 17874 raft_consensus.cc:3037] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:29.878502 17874 raft_consensus.cc:2454] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 451db10510a9469b92abc0705d7bf44d in term 2.
I20240502 07:58:29.879586 19207 leader_election.cc:302] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 3d36406429504d36ab670d84f4464efa, 451db10510a9469b92abc0705d7bf44d; no voters:
I20240502 07:58:29.879868 20477 raft_consensus.cc:2781] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:58:29.880064 20477 raft_consensus.cc:686] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d [term 2 LEADER]: Becoming Leader. State: Replica: 451db10510a9469b92abc0705d7bf44d, State: Running, Role: LEADER
I20240502 07:58:29.880259 20477 consensus_queue.cc:237] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } }
I20240502 07:58:29.884047 17732 catalog_manager.cc:5484] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d reported cstate change: term changed from 1 to 2, leader changed from 3d36406429504d36ab670d84f4464efa (127.15.129.129) to 451db10510a9469b92abc0705d7bf44d (127.15.129.133). New cstate: current_term: 2 leader_uuid: "451db10510a9469b92abc0705d7bf44d" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } } }
I20240502 07:58:29.886368 17984 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:58:29.887215 17853 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:58:29.889271 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:29.892187 19123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:29.901350 19395 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:29.914394 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:29.915138 18992 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:30.004429 17853 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:58:30.004853 19123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:30.006397 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.007483 18992 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:30.019580 17984 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:58:30.026402 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.026420 19395 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:30.033841 18005 raft_consensus.cc:1261] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Refusing update from remote peer 9fdc864a7d9349808a6fc7b2cf4eba8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:58:30.034571 20517 consensus_queue.cc:1035] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:30.058158 18005 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a7a2522085124c63a938200b3e925cdb"
dest_uuid: "923d7729977f4edda0aa760e184a8830"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37472
I20240502 07:58:30.058331 18005 raft_consensus.cc:595] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:58:30.075553 17874 raft_consensus.cc:1261] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Refusing update from remote peer d87eea2501954fca83327b9408429b66: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:58:30.075984 20533 consensus_queue.cc:1035] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:30.112903 17853 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:58:30.113466 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.113734 19123 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:30.116503 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.120433 18992 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:30.142866 17984 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:58:30.149858 19395 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:30.154707 19852 raft_consensus.cc:979] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa: : Instructing follower 3f7795cc262b43cfa8a9252c424b7354 to start an election
I20240502 07:58:30.154851 19852 raft_consensus.cc:1067] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Signalling peer 3f7795cc262b43cfa8a9252c424b7354 to start an election
I20240502 07:58:30.155349 19143 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "62ba9d55df884c568ea6285701011ceb"
dest_uuid: "3f7795cc262b43cfa8a9252c424b7354"
from {username='slave'} at 127.15.129.129:60743
I20240502 07:58:30.155566 19143 raft_consensus.cc:482] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:58:30.155675 19143 raft_consensus.cc:3037] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:30.156811 19143 raft_consensus.cc:504] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } }
I20240502 07:58:30.157876 19143 leader_election.cc:288] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 [CANDIDATE]: Term 2 election: Requested vote from peers 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343)
I20240502 07:58:30.158322 17874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ba9d55df884c568ea6285701011ceb" candidate_uuid: "3f7795cc262b43cfa8a9252c424b7354" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d36406429504d36ab670d84f4464efa"
I20240502 07:58:30.158457 17874 raft_consensus.cc:3032] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:58:30.158501 17874 raft_consensus.cc:729] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Running, Role: LEADER
I20240502 07:58:30.158634 17874 consensus_queue.cc:260] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } }
I20240502 07:58:30.158716 17874 raft_consensus.cc:3037] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:30.159719 17874 raft_consensus.cc:2454] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f7795cc262b43cfa8a9252c424b7354 in term 2.
I20240502 07:58:30.160303 19076 leader_election.cc:302] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 3d36406429504d36ab670d84f4464efa, 3f7795cc262b43cfa8a9252c424b7354; no voters:
I20240502 07:58:30.160650 20563 raft_consensus.cc:2781] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:58:30.160840 20563 raft_consensus.cc:686] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 [term 2 LEADER]: Becoming Leader. State: Replica: 3f7795cc262b43cfa8a9252c424b7354, State: Running, Role: LEADER
I20240502 07:58:30.161164 20563 consensus_queue.cc:237] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } }
I20240502 07:58:30.163357 17728 catalog_manager.cc:5484] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 reported cstate change: term changed from 1 to 2, leader changed from 3d36406429504d36ab670d84f4464efa (127.15.129.129) to 3f7795cc262b43cfa8a9252c424b7354 (127.15.129.132). New cstate: current_term: 2 leader_uuid: "3f7795cc262b43cfa8a9252c424b7354" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } } }
I20240502 07:58:30.228765 17853 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:30.230196 19123 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:30.233749 18992 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:30.235255 19395 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:30.245882 17984 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:58:30.247695 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.249755 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.278496 18005 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4ae9214f8e5a4113b339c78cf80b1ed8"
dest_uuid: "923d7729977f4edda0aa760e184a8830"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:37494
I20240502 07:58:30.278645 18005 raft_consensus.cc:595] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:58:30.280684 20615 raft_consensus.cc:979] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830: : Instructing follower 9fdc864a7d9349808a6fc7b2cf4eba8f to start an election
I20240502 07:58:30.280782 20615 raft_consensus.cc:1067] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Signalling peer 9fdc864a7d9349808a6fc7b2cf4eba8f to start an election
I20240502 07:58:30.281245 19012 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a7a2522085124c63a938200b3e925cdb"
dest_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f"
from {username='slave'} at 127.15.129.130:37077
I20240502 07:58:30.281414 19012 raft_consensus.cc:482] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:58:30.281488 19012 raft_consensus.cc:3037] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:30.282284 19012 raft_consensus.cc:504] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } }
I20240502 07:58:30.282580 19012 leader_election.cc:288] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f [CANDIDATE]: Term 2 election: Requested vote from peers 923d7729977f4edda0aa760e184a8830 (127.15.129.130:45189)
I20240502 07:58:30.283044 18005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7a2522085124c63a938200b3e925cdb" candidate_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "923d7729977f4edda0aa760e184a8830"
I20240502 07:58:30.283162 18005 raft_consensus.cc:3032] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:58:30.283210 18005 raft_consensus.cc:729] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Running, Role: LEADER
I20240502 07:58:30.283332 18005 consensus_queue.cc:260] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } }
I20240502 07:58:30.283453 18005 raft_consensus.cc:3037] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:30.284224 18005 raft_consensus.cc:2454] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fdc864a7d9349808a6fc7b2cf4eba8f in term 2.
I20240502 07:58:30.284566 18947 leader_election.cc:302] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 923d7729977f4edda0aa760e184a8830, 9fdc864a7d9349808a6fc7b2cf4eba8f; no voters:
I20240502 07:58:30.284832 20519 raft_consensus.cc:2781] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:58:30.284929 20519 raft_consensus.cc:686] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 2 LEADER]: Becoming Leader. State: Replica: 9fdc864a7d9349808a6fc7b2cf4eba8f, State: Running, Role: LEADER
I20240502 07:58:30.285059 20519 consensus_queue.cc:237] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } }
I20240502 07:58:30.286634 17732 catalog_manager.cc:5484] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f reported cstate change: term changed from 1 to 2, leader changed from 923d7729977f4edda0aa760e184a8830 (127.15.129.130) to 9fdc864a7d9349808a6fc7b2cf4eba8f (127.15.129.131). New cstate: current_term: 2 leader_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } } }
I20240502 07:58:30.334491 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.337354 17853 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:30.338174 17984 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:30.338567 19395 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:30.343720 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.351076 19123 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:30.373245 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.453241 17853 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:30.454286 19123 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:30.455781 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.457239 19395 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:30.461661 17984 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:30.463279 17874 raft_consensus.cc:1261] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Refusing update from remote peer 451db10510a9469b92abc0705d7bf44d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:58:30.463765 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.463857 20680 consensus_queue.cc:1035] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:30.465782 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.506764 17874 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "323b33d450384c90b13a1ac565f411a4"
dest_uuid: "3d36406429504d36ab670d84f4464efa"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:47044
I20240502 07:58:30.506907 17874 raft_consensus.cc:595] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Received request to transfer leadership
I20240502 07:58:30.563774 17853 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:30.563974 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.564579 19123 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:30.565014 17984 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:30.566395 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.585120 17874 raft_consensus.cc:1261] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Refusing update from remote peer 3f7795cc262b43cfa8a9252c424b7354: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:58:30.585281 19395 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:30.585695 20564 consensus_queue.cc:1035] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:30.604650 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.792163 20723 raft_consensus.cc:979] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830: : Instructing follower c815aa98415f4682b2c8429a67e85444 to start an election
I20240502 07:58:30.792276 20723 raft_consensus.cc:1067] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Signalling peer c815aa98415f4682b2c8429a67e85444 to start an election
I20240502 07:58:30.792842 19415 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4ae9214f8e5a4113b339c78cf80b1ed8"
dest_uuid: "c815aa98415f4682b2c8429a67e85444"
from {username='slave'} at 127.15.129.130:48123
I20240502 07:58:30.792982 19415 raft_consensus.cc:482] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:58:30.793032 19415 raft_consensus.cc:3037] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:30.793965 19415 raft_consensus.cc:504] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } }
I20240502 07:58:30.794360 19415 leader_election.cc:288] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444 [CANDIDATE]: Term 2 election: Requested vote from peers 923d7729977f4edda0aa760e184a8830 (127.15.129.130:45189)
I20240502 07:58:30.794775 18005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ae9214f8e5a4113b339c78cf80b1ed8" candidate_uuid: "c815aa98415f4682b2c8429a67e85444" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "923d7729977f4edda0aa760e184a8830"
I20240502 07:58:30.794875 18005 raft_consensus.cc:3032] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:58:30.794910 18005 raft_consensus.cc:729] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Running, Role: LEADER
I20240502 07:58:30.795058 18005 consensus_queue.cc:260] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } }
I20240502 07:58:30.795197 18005 raft_consensus.cc:3037] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:30.795908 18005 raft_consensus.cc:2454] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c815aa98415f4682b2c8429a67e85444 in term 2.
I20240502 07:58:30.796219 19350 leader_election.cc:302] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 923d7729977f4edda0aa760e184a8830, c815aa98415f4682b2c8429a67e85444; no voters:
I20240502 07:58:30.796432 20582 raft_consensus.cc:2781] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:58:30.796627 20582 raft_consensus.cc:686] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444 [term 2 LEADER]: Becoming Leader. State: Replica: c815aa98415f4682b2c8429a67e85444, State: Running, Role: LEADER
I20240502 07:58:30.796797 20582 consensus_queue.cc:237] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } }
I20240502 07:58:30.798472 17729 catalog_manager.cc:5484] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444 reported cstate change: term changed from 1 to 2, leader changed from 923d7729977f4edda0aa760e184a8830 (127.15.129.130) to c815aa98415f4682b2c8429a67e85444 (127.15.129.134). New cstate: current_term: 2 leader_uuid: "c815aa98415f4682b2c8429a67e85444" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } } }
I20240502 07:58:30.834466 18005 raft_consensus.cc:1261] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Refusing update from remote peer 9fdc864a7d9349808a6fc7b2cf4eba8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:58:30.834914 20519 consensus_queue.cc:1035] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:30.882041 19123 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:30.888609 17853 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:30.891481 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.898499 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.900426 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:30.906679 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.924770 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:30.963009 19415 consensus_queue.cc:237] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } }
I20240502 07:58:30.963899 20582 raft_consensus.cc:2932] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 3d36406429504d36ab670d84f4464efa (127.15.129.129) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } } }
I20240502 07:58:30.966518 17729 catalog_manager.cc:5484] T 4fb3087aff7d40eb960135010402371e P c815aa98415f4682b2c8429a67e85444 reported cstate change: config changed from index 4 to 6, VOTER 3d36406429504d36ab670d84f4464efa (127.15.129.129) evicted. New cstate: current_term: 2 leader_uuid: "c815aa98415f4682b2c8429a67e85444" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } } }
I20240502 07:58:30.969185 17853 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fb3087aff7d40eb960135010402371e with delete_type TABLET_DATA_TOMBSTONED (TS 3d36406429504d36ab670d84f4464efa not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:50568
I20240502 07:58:30.969683 20792 tablet_replica.cc:330] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa: stopping tablet replica
I20240502 07:58:30.970017 20792 raft_consensus.cc:2229] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:30.970198 20792 raft_consensus.cc:2258] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:30.971148 20792 ts_tablet_manager.cc:1875] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:30.973567 20792 ts_tablet_manager.cc:1888] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 07:58:30.973650 20792 log.cc:1196] T 4fb3087aff7d40eb960135010402371e P 3d36406429504d36ab670d84f4464efa: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/4fb3087aff7d40eb960135010402371e
I20240502 07:58:30.974442 17716 catalog_manager.cc:4830] TS 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343): tablet 4fb3087aff7d40eb960135010402371e (table rebalance_test_table_0 [id=cb32abd9e63d47ffa9ab6f1229f6279c]) successfully deleted
I20240502 07:58:31.019783 19852 raft_consensus.cc:979] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa: : Instructing follower 3f7795cc262b43cfa8a9252c424b7354 to start an election
I20240502 07:58:31.019896 19852 raft_consensus.cc:1067] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Signalling peer 3f7795cc262b43cfa8a9252c424b7354 to start an election
I20240502 07:58:31.020296 19143 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "323b33d450384c90b13a1ac565f411a4"
dest_uuid: "3f7795cc262b43cfa8a9252c424b7354"
from {username='slave'} at 127.15.129.129:60743
I20240502 07:58:31.020469 19143 raft_consensus.cc:482] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:58:31.020545 19143 raft_consensus.cc:3037] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:31.021649 19143 raft_consensus.cc:504] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } }
I20240502 07:58:31.021948 19143 leader_election.cc:288] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354 [CANDIDATE]: Term 2 election: Requested vote from peers 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343)
I20240502 07:58:31.022287 17874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "323b33d450384c90b13a1ac565f411a4" candidate_uuid: "3f7795cc262b43cfa8a9252c424b7354" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d36406429504d36ab670d84f4464efa"
I20240502 07:58:31.022440 17874 raft_consensus.cc:3032] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:58:31.022486 17874 raft_consensus.cc:729] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Running, Role: LEADER
I20240502 07:58:31.022640 17874 consensus_queue.cc:260] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } }
I20240502 07:58:31.022780 17874 raft_consensus.cc:3037] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:31.023911 17874 raft_consensus.cc:2454] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f7795cc262b43cfa8a9252c424b7354 in term 2.
I20240502 07:58:31.024221 19076 leader_election.cc:302] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 3d36406429504d36ab670d84f4464efa, 3f7795cc262b43cfa8a9252c424b7354; no voters:
I20240502 07:58:31.024706 20563 raft_consensus.cc:2781] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:58:31.024798 20563 raft_consensus.cc:686] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354 [term 2 LEADER]: Becoming Leader. State: Replica: 3f7795cc262b43cfa8a9252c424b7354, State: Running, Role: LEADER
I20240502 07:58:31.024936 20563 consensus_queue.cc:237] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } }
I20240502 07:58:31.026809 17727 catalog_manager.cc:5484] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354 reported cstate change: term changed from 1 to 2, leader changed from 3d36406429504d36ab670d84f4464efa (127.15.129.129) to 3f7795cc262b43cfa8a9252c424b7354 (127.15.129.132). New cstate: current_term: 2 leader_uuid: "3f7795cc262b43cfa8a9252c424b7354" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } } }
I20240502 07:58:31.032065 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.032982 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.034176 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.035676 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.036125 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:31.041954 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.042404 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:31.094702 19546 consensus_queue.cc:237] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } }
I20240502 07:58:31.095484 20535 raft_consensus.cc:2932] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 923d7729977f4edda0aa760e184a8830 (127.15.129.130) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } } }
I20240502 07:58:31.097764 17727 catalog_manager.cc:5484] T 3f6c4761aef54c538693f1d3a4d80d10 P d87eea2501954fca83327b9408429b66 reported cstate change: config changed from index 4 to 6, VOTER 923d7729977f4edda0aa760e184a8830 (127.15.129.130) evicted. New cstate: current_term: 2 leader_uuid: "d87eea2501954fca83327b9408429b66" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } } }
I20240502 07:58:31.100461 17984 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f6c4761aef54c538693f1d3a4d80d10 with delete_type TABLET_DATA_TOMBSTONED (TS 923d7729977f4edda0aa760e184a8830 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45534
I20240502 07:58:31.100914 20850 tablet_replica.cc:330] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830: stopping tablet replica
I20240502 07:58:31.101240 20850 raft_consensus.cc:2229] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:31.101444 20850 raft_consensus.cc:2258] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:31.102397 20850 ts_tablet_manager.cc:1875] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:31.105381 20850 ts_tablet_manager.cc:1888] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 07:58:31.105506 20850 log.cc:1196] T 3f6c4761aef54c538693f1d3a4d80d10 P 923d7729977f4edda0aa760e184a8830: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/3f6c4761aef54c538693f1d3a4d80d10
I20240502 07:58:31.106384 17718 catalog_manager.cc:4830] TS 923d7729977f4edda0aa760e184a8830 (127.15.129.130:45189): tablet 3f6c4761aef54c538693f1d3a4d80d10 (table rebalance_test_table_1 [id=660ea085db6f4a49a3d523727eb2b72f]) successfully deleted
I20240502 07:58:31.162591 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:31.164314 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:31.165583 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.167565 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.169654 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.172624 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.180613 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.232343 19012 consensus_queue.cc:237] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } }
I20240502 07:58:31.233103 20742 raft_consensus.cc:2932] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 923d7729977f4edda0aa760e184a8830 (127.15.129.130) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } } }
I20240502 07:58:31.235749 17731 catalog_manager.cc:5484] T e82159be5de549fe9ed93db5ef6262fc P 9fdc864a7d9349808a6fc7b2cf4eba8f reported cstate change: config changed from index 4 to 6, VOTER 923d7729977f4edda0aa760e184a8830 (127.15.129.130) evicted. New cstate: current_term: 2 leader_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } } }
I20240502 07:58:31.238317 17984 tablet_service.cc:1514] Processing DeleteTablet for tablet e82159be5de549fe9ed93db5ef6262fc with delete_type TABLET_DATA_TOMBSTONED (TS 923d7729977f4edda0aa760e184a8830 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45534
I20240502 07:58:31.238528 20850 tablet_replica.cc:330] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830: stopping tablet replica
I20240502 07:58:31.238646 20850 raft_consensus.cc:2229] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:31.238746 20850 raft_consensus.cc:2258] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:31.239291 20850 ts_tablet_manager.cc:1875] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:31.241513 20850 ts_tablet_manager.cc:1888] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 07:58:31.241580 20850 log.cc:1196] T e82159be5de549fe9ed93db5ef6262fc P 923d7729977f4edda0aa760e184a8830: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/e82159be5de549fe9ed93db5ef6262fc
I20240502 07:58:31.242079 17718 catalog_manager.cc:4830] TS 923d7729977f4edda0aa760e184a8830 (127.15.129.130:45189): tablet e82159be5de549fe9ed93db5ef6262fc (table rebalance_test_table_3 [id=30cbd54b96274072a006ed7c32764da1]) successfully deleted
I20240502 07:58:31.294484 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:31.295257 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.296417 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.298753 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.302771 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:31.317075 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.321990 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.361096 19546 consensus_queue.cc:237] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } }
I20240502 07:58:31.361757 20535 raft_consensus.cc:2932] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 3d36406429504d36ab670d84f4464efa (127.15.129.129) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } } }
I20240502 07:58:31.364023 17732 catalog_manager.cc:5484] T bab58d3c92774b1ab9fa77e100db9073 P d87eea2501954fca83327b9408429b66 reported cstate change: config changed from index 4 to 6, VOTER 3d36406429504d36ab670d84f4464efa (127.15.129.129) evicted. New cstate: current_term: 2 leader_uuid: "d87eea2501954fca83327b9408429b66" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d87eea2501954fca83327b9408429b66" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 37247 } } }
I20240502 07:58:31.367194 17853 tablet_service.cc:1514] Processing DeleteTablet for tablet bab58d3c92774b1ab9fa77e100db9073 with delete_type TABLET_DATA_TOMBSTONED (TS 3d36406429504d36ab670d84f4464efa not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:50568
I20240502 07:58:31.367470 20792 tablet_replica.cc:330] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa: stopping tablet replica
I20240502 07:58:31.367611 20792 raft_consensus.cc:2229] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:31.367741 20792 raft_consensus.cc:2258] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:31.368324 20792 ts_tablet_manager.cc:1875] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:31.370366 20792 ts_tablet_manager.cc:1888] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 07:58:31.370455 20792 log.cc:1196] T bab58d3c92774b1ab9fa77e100db9073 P 3d36406429504d36ab670d84f4464efa: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/bab58d3c92774b1ab9fa77e100db9073
I20240502 07:58:31.371002 17716 catalog_manager.cc:4830] TS 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343): tablet bab58d3c92774b1ab9fa77e100db9073 (table rebalance_test_table_3 [id=30cbd54b96274072a006ed7c32764da1]) successfully deleted
I20240502 07:58:31.401656 18005 raft_consensus.cc:1261] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Refusing update from remote peer c815aa98415f4682b2c8429a67e85444: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:58:31.402163 20582 consensus_queue.cc:1035] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444 [LEADER]: Connected to new peer: Peer: permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:31.426470 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:31.427466 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.430245 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:31.434168 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.437742 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.441494 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.441912 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.488857 19275 consensus_queue.cc:237] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } }
I20240502 07:58:31.489480 20680 raft_consensus.cc:2932] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 3d36406429504d36ab670d84f4464efa (127.15.129.129) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } } }
I20240502 07:58:31.491643 17731 catalog_manager.cc:5484] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 451db10510a9469b92abc0705d7bf44d reported cstate change: config changed from index 4 to 6, VOTER 3d36406429504d36ab670d84f4464efa (127.15.129.129) evicted. New cstate: current_term: 2 leader_uuid: "451db10510a9469b92abc0705d7bf44d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "451db10510a9469b92abc0705d7bf44d" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 34385 } } }
I20240502 07:58:31.494385 17853 tablet_service.cc:1514] Processing DeleteTablet for tablet f49ef9b8bef141a3bc00ec7bf7c1ac67 with delete_type TABLET_DATA_TOMBSTONED (TS 3d36406429504d36ab670d84f4464efa not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:50568
I20240502 07:58:31.494643 20792 tablet_replica.cc:330] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa: stopping tablet replica
I20240502 07:58:31.494797 20792 raft_consensus.cc:2229] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:31.494930 20792 raft_consensus.cc:2258] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:31.495586 20792 ts_tablet_manager.cc:1875] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:31.497856 20792 ts_tablet_manager.cc:1888] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 07:58:31.497949 20792 log.cc:1196] T f49ef9b8bef141a3bc00ec7bf7c1ac67 P 3d36406429504d36ab670d84f4464efa: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/f49ef9b8bef141a3bc00ec7bf7c1ac67
I20240502 07:58:31.498514 17716 catalog_manager.cc:4830] TS 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343): tablet f49ef9b8bef141a3bc00ec7bf7c1ac67 (table rebalance_test_table_1 [id=660ea085db6f4a49a3d523727eb2b72f]) successfully deleted
I20240502 07:58:31.517901 17873 raft_consensus.cc:1261] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Refusing update from remote peer 3f7795cc262b43cfa8a9252c424b7354: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:58:31.518391 20563 consensus_queue.cc:1035] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:31.553313 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.554378 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.554447 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:31.557672 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:31.566597 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.579981 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.582319 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.617440 19143 consensus_queue.cc:237] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } }
I20240502 07:58:31.618192 20564 raft_consensus.cc:2932] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 3d36406429504d36ab670d84f4464efa (127.15.129.129) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } } }
I20240502 07:58:31.620376 17731 catalog_manager.cc:5484] T 62ba9d55df884c568ea6285701011ceb P 3f7795cc262b43cfa8a9252c424b7354 reported cstate change: config changed from index 4 to 6, VOTER 3d36406429504d36ab670d84f4464efa (127.15.129.129) evicted. New cstate: current_term: 2 leader_uuid: "3f7795cc262b43cfa8a9252c424b7354" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } } }
I20240502 07:58:31.623222 17853 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ba9d55df884c568ea6285701011ceb with delete_type TABLET_DATA_TOMBSTONED (TS 3d36406429504d36ab670d84f4464efa not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:50568
I20240502 07:58:31.623497 20792 tablet_replica.cc:330] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa: stopping tablet replica
I20240502 07:58:31.623656 20792 raft_consensus.cc:2229] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:31.623777 20792 raft_consensus.cc:2258] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:31.624262 20792 ts_tablet_manager.cc:1875] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:31.626098 20792 ts_tablet_manager.cc:1888] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 07:58:31.626180 20792 log.cc:1196] T 62ba9d55df884c568ea6285701011ceb P 3d36406429504d36ab670d84f4464efa: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/62ba9d55df884c568ea6285701011ceb
I20240502 07:58:31.626660 17716 catalog_manager.cc:4830] TS 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343): tablet 62ba9d55df884c568ea6285701011ceb (table rebalance_test_table_2 [id=46e1dc9f151748c996c37174bb27644c]) successfully deleted
I20240502 07:58:31.682193 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:31.684978 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.685865 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:31.687340 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.687489 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.689385 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.689819 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.750414 19012 consensus_queue.cc:237] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } }
I20240502 07:58:31.750969 20742 raft_consensus.cc:2932] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 923d7729977f4edda0aa760e184a8830 (127.15.129.130) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } } }
I20240502 07:58:31.753302 17732 catalog_manager.cc:5484] T a7a2522085124c63a938200b3e925cdb P 9fdc864a7d9349808a6fc7b2cf4eba8f reported cstate change: config changed from index 4 to 6, VOTER 923d7729977f4edda0aa760e184a8830 (127.15.129.130) evicted. New cstate: current_term: 2 leader_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9fdc864a7d9349808a6fc7b2cf4eba8f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43267 } } }
I20240502 07:58:31.755841 17984 tablet_service.cc:1514] Processing DeleteTablet for tablet a7a2522085124c63a938200b3e925cdb with delete_type TABLET_DATA_TOMBSTONED (TS 923d7729977f4edda0aa760e184a8830 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45534
I20240502 07:58:31.756166 21124 tablet_replica.cc:330] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830: stopping tablet replica
I20240502 07:58:31.756290 21124 raft_consensus.cc:2229] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:31.756397 21124 raft_consensus.cc:2258] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:31.757006 21124 ts_tablet_manager.cc:1875] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:31.759181 21124 ts_tablet_manager.cc:1888] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 07:58:31.759261 21124 log.cc:1196] T a7a2522085124c63a938200b3e925cdb P 923d7729977f4edda0aa760e184a8830: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/a7a2522085124c63a938200b3e925cdb
I20240502 07:58:31.759799 17718 catalog_manager.cc:4830] TS 923d7729977f4edda0aa760e184a8830 (127.15.129.130:45189): tablet a7a2522085124c63a938200b3e925cdb (table rebalance_test_table_0 [id=cb32abd9e63d47ffa9ab6f1229f6279c]) successfully deleted
I20240502 07:58:31.815047 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:31.815132 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:31.819031 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.819717 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.825515 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.834143 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.842975 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.880429 19415 consensus_queue.cc:237] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } }
I20240502 07:58:31.881031 20725 raft_consensus.cc:2932] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 923d7729977f4edda0aa760e184a8830 (127.15.129.130) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } } }
I20240502 07:58:31.883350 17728 catalog_manager.cc:5484] T 4ae9214f8e5a4113b339c78cf80b1ed8 P c815aa98415f4682b2c8429a67e85444 reported cstate change: config changed from index 4 to 6, VOTER 923d7729977f4edda0aa760e184a8830 (127.15.129.130) evicted. New cstate: current_term: 2 leader_uuid: "c815aa98415f4682b2c8429a67e85444" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c815aa98415f4682b2c8429a67e85444" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 34275 } } }
I20240502 07:58:31.885687 17984 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ae9214f8e5a4113b339c78cf80b1ed8 with delete_type TABLET_DATA_TOMBSTONED (TS 923d7729977f4edda0aa760e184a8830 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45534
I20240502 07:58:31.885958 21124 tablet_replica.cc:330] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830: stopping tablet replica
I20240502 07:58:31.886076 21124 raft_consensus.cc:2229] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:31.886165 21124 raft_consensus.cc:2258] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:31.886744 21124 ts_tablet_manager.cc:1875] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:31.888571 21124 ts_tablet_manager.cc:1888] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 07:58:31.888643 21124 log.cc:1196] T 4ae9214f8e5a4113b339c78cf80b1ed8 P 923d7729977f4edda0aa760e184a8830: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/4ae9214f8e5a4113b339c78cf80b1ed8
I20240502 07:58:31.889084 17718 catalog_manager.cc:4830] TS 923d7729977f4edda0aa760e184a8830 (127.15.129.130:45189): tablet 4ae9214f8e5a4113b339c78cf80b1ed8 (table rebalance_test_table_2 [id=46e1dc9f151748c996c37174bb27644c]) successfully deleted
I20240502 07:58:31.943065 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:31.943331 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.945269 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.954403 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:31.957638 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.965232 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:31.965979 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.008347 19143 consensus_queue.cc:237] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } }
I20240502 07:58:32.009020 20564 raft_consensus.cc:2932] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 3d36406429504d36ab670d84f4464efa (127.15.129.129) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } } }
I20240502 07:58:32.011363 17731 catalog_manager.cc:5484] T 323b33d450384c90b13a1ac565f411a4 P 3f7795cc262b43cfa8a9252c424b7354 reported cstate change: config changed from index 4 to 6, VOTER 3d36406429504d36ab670d84f4464efa (127.15.129.129) evicted. New cstate: current_term: 2 leader_uuid: "3f7795cc262b43cfa8a9252c424b7354" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3f7795cc262b43cfa8a9252c424b7354" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 43973 } } }
I20240502 07:58:32.013651 17853 tablet_service.cc:1514] Processing DeleteTablet for tablet 323b33d450384c90b13a1ac565f411a4 with delete_type TABLET_DATA_TOMBSTONED (TS 3d36406429504d36ab670d84f4464efa not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:50568
I20240502 07:58:32.013885 20792 tablet_replica.cc:330] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa: stopping tablet replica
I20240502 07:58:32.014024 20792 raft_consensus.cc:2229] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:32.014132 20792 raft_consensus.cc:2258] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:32.014765 20792 ts_tablet_manager.cc:1875] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:32.016568 20792 ts_tablet_manager.cc:1888] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 07:58:32.016639 20792 log.cc:1196] T 323b33d450384c90b13a1ac565f411a4 P 3d36406429504d36ab670d84f4464efa: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/323b33d450384c90b13a1ac565f411a4
I20240502 07:58:32.017171 17716 catalog_manager.cc:4830] TS 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343): tablet 323b33d450384c90b13a1ac565f411a4 (table rebalance_test_table_4 [id=aa24b74e8feb4d10b47ce377487513ca]) successfully deleted
I20240502 07:58:32.316428 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.316576 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:32.319042 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:32.325690 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.329878 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.341101 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.341744 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.428906 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.430506 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:32.432778 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:32.432880 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.434526 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.446321 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.452783 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.675390 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.676612 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:32.678555 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.682263 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.686623 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:32.698470 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.698900 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.810477 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:32.812072 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:32.812772 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.814744 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.818130 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.833534 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.837481 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.936518 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.938925 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.941788 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.948441 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:32.948544 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:32.948843 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:32.957171 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
53be75cf33b74f38884080e1936c65f1 | 127.15.129.190:41083 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+----------------------+-------------------------
builtin_ntp_servers | 127.15.129.148:41341 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
3d36406429504d36ab670d84f4464efa | 127.15.129.129:42343 | HEALTHY | <none> | 3 | 0
3f7795cc262b43cfa8a9252c424b7354 | 127.15.129.132:43973 | HEALTHY | <none> | 2 | 0
451db10510a9469b92abc0705d7bf44d | 127.15.129.133:34385 | HEALTHY | <none> | 2 | 0
923d7729977f4edda0aa760e184a8830 | 127.15.129.130:45189 | HEALTHY | <none> | 3 | 0
9fdc864a7d9349808a6fc7b2cf4eba8f | 127.15.129.131:43267 | HEALTHY | <none> | 2 | 0
c815aa98415f4682b2c8429a67e85444 | 127.15.129.134:34275 | HEALTHY | <none> | 2 | 0
d87eea2501954fca83327b9408429b66 | 127.15.129.135:37247 | HEALTHY | <none> | 2 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 7
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 7 server(s) checked
local_ip_for_outbound_sockets | 127.15.129.129 | experimental | 127.15.129.129:42343
local_ip_for_outbound_sockets | 127.15.129.130 | experimental | 127.15.129.130:45189
local_ip_for_outbound_sockets | 127.15.129.131 | experimental | 127.15.129.131:43267
local_ip_for_outbound_sockets | 127.15.129.132 | experimental | 127.15.129.132:43973
local_ip_for_outbound_sockets | 127.15.129.133 | experimental | 127.15.129.133:34385
local_ip_for_outbound_sockets | 127.15.129.134 | experimental | 127.15.129.134:34275
local_ip_for_outbound_sockets | 127.15.129.135 | experimental | 127.15.129.135:37247
never_fsync | true | unsafe,advanced | all 7 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 7 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 7 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 7 server(s) checked
server_dump_info_format | pb | hidden | all 7 server(s) checked
server_dump_info_path | /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.15.129.129:42343
server_dump_info_path | /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.15.129.130:45189
server_dump_info_path | /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.15.129.131:43267
server_dump_info_path | /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.15.129.132:43973
server_dump_info_path | /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.15.129.133:34385
server_dump_info_path | /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.15.129.134:34275
server_dump_info_path | /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.15.129.135:37247
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+----------------------+-------------------------
builtin_ntp_servers | 127.15.129.148:41341 | all 7 server(s) checked
time_source | builtin | all 7 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 8 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 2
First Quartile | 2
Median | 2
Third Quartile | 3
Maximum | 3
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 7
Tables | 6
Tablets | 16
Replicas | 16
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240502 07:58:32.990996 15878 log_verifier.cc:126] Checking tablet 04638eb3c88040f7b40f7f6ec6989e06
I20240502 07:58:33.001965 15878 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 04638eb3c88040f7b40f7f6ec6989e06
I20240502 07:58:33.002046 15878 log_verifier.cc:126] Checking tablet 1042d2331b5a425eb8e6721cb5c4b894
I20240502 07:58:33.010252 15878 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 1042d2331b5a425eb8e6721cb5c4b894
I20240502 07:58:33.010320 15878 log_verifier.cc:126] Checking tablet 323b33d450384c90b13a1ac565f411a4
I20240502 07:58:33.018829 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 323b33d450384c90b13a1ac565f411a4
I20240502 07:58:33.018923 15878 log_verifier.cc:126] Checking tablet 334047bb4be641e0aad4f6405e067468
I20240502 07:58:33.027608 15878 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 334047bb4be641e0aad4f6405e067468
I20240502 07:58:33.027674 15878 log_verifier.cc:126] Checking tablet 3f6c4761aef54c538693f1d3a4d80d10
I20240502 07:58:33.036912 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 3f6c4761aef54c538693f1d3a4d80d10
I20240502 07:58:33.036993 15878 log_verifier.cc:126] Checking tablet 41a4e8bf2f494f159b25a8437f7bdeec
I20240502 07:58:33.045400 15878 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 41a4e8bf2f494f159b25a8437f7bdeec
I20240502 07:58:33.045472 15878 log_verifier.cc:126] Checking tablet 4609595bd09d4893b2575ca6a2740896
I20240502 07:58:33.054630 15878 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 4609595bd09d4893b2575ca6a2740896
I20240502 07:58:33.054695 15878 log_verifier.cc:126] Checking tablet 4ae9214f8e5a4113b339c78cf80b1ed8
I20240502 07:58:33.063638 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 4ae9214f8e5a4113b339c78cf80b1ed8
I20240502 07:58:33.063704 15878 log_verifier.cc:126] Checking tablet 4fb3087aff7d40eb960135010402371e
I20240502 07:58:33.072947 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 4fb3087aff7d40eb960135010402371e
I20240502 07:58:33.073011 15878 log_verifier.cc:126] Checking tablet 62ba9d55df884c568ea6285701011ceb
I20240502 07:58:33.082185 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 62ba9d55df884c568ea6285701011ceb
I20240502 07:58:33.082248 15878 log_verifier.cc:126] Checking tablet 706476e790b54b2690ea892fb6850046
I20240502 07:58:33.091291 15878 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 706476e790b54b2690ea892fb6850046
I20240502 07:58:33.091352 15878 log_verifier.cc:126] Checking tablet 9d165c26beff4f3dacf61ca92fdec35f
I20240502 07:58:33.100544 15878 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 9d165c26beff4f3dacf61ca92fdec35f
I20240502 07:58:33.100611 15878 log_verifier.cc:126] Checking tablet a7a2522085124c63a938200b3e925cdb
I20240502 07:58:33.110291 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet a7a2522085124c63a938200b3e925cdb
I20240502 07:58:33.110354 15878 log_verifier.cc:126] Checking tablet bab58d3c92774b1ab9fa77e100db9073
I20240502 07:58:33.119920 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet bab58d3c92774b1ab9fa77e100db9073
I20240502 07:58:33.120038 15878 log_verifier.cc:126] Checking tablet e82159be5de549fe9ed93db5ef6262fc
I20240502 07:58:33.129638 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet e82159be5de549fe9ed93db5ef6262fc
I20240502 07:58:33.129707 15878 log_verifier.cc:126] Checking tablet f49ef9b8bef141a3bc00ec7bf7c1ac67
I20240502 07:58:33.139305 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet f49ef9b8bef141a3bc00ec7bf7c1ac67
I20240502 07:58:33.141444 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.136:0
--local_ip_for_outbound_sockets=127.15.129.136
--webserver_interface=127.15.129.136
--webserver_port=0
--tserver_master_addrs=127.15.129.190:41083
--builtin_ntp_servers=127.15.129.148:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 07:58:33.252846 21440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:33.253119 21440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:33.253168 21440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:33.256397 21440 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:33.256464 21440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:33.256536 21440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.136
I20240502 07:58:33.260185 21440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.136:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.15.129.136
--webserver_port=0
--tserver_master_addrs=127.15.129.190:41083
--never_fsync=true
--heap_profile_path=/tmp/kudu.21440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.136
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:33.261065 21440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:33.262199 21440 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240502 07:58:33.271688 21440 server_base.cc:1034] running on GCE node
W20240502 07:58:33.273109 21448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:33.273522 21445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:33.274520 21446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:33.274943 21440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:33.275513 21440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:33.276659 21440 hybrid_clock.cc:648] HybridClock initialized: now 1714636713276647 us; error 30 us; skew 500 ppm
I20240502 07:58:33.278530 21440 webserver.cc:460] Webserver started at http://127.15.129.136:43231/ using document root <none> and password file <none>
I20240502 07:58:33.279063 21440 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:33.279131 21440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:33.279299 21440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:33.280891 21440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "a0d44a6c162042df9028884a953767ff"
format_stamp: "Formatted at 2024-05-02 07:58:33 on dist-test-slave-0hx2"
I20240502 07:58:33.281368 21440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "a0d44a6c162042df9028884a953767ff"
format_stamp: "Formatted at 2024-05-02 07:58:33 on dist-test-slave-0hx2"
I20240502 07:58:33.284601 21440 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20240502 07:58:33.286883 21454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:33.288007 21440 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:33.288110 21440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "a0d44a6c162042df9028884a953767ff"
format_stamp: "Formatted at 2024-05-02 07:58:33 on dist-test-slave-0hx2"
I20240502 07:58:33.288208 21440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:33.313984 21440 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:33.314659 21440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:33.314824 21440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:33.315362 21440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:33.316269 21440 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:33.316319 21440 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:33.316383 21440 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:33.316411 21440 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:33.325542 21440 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.136:33243
I20240502 07:58:33.325560 21567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.136:33243 every 8 connection(s)
I20240502 07:58:33.326464 21440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 07:58:33.327638 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 21440
I20240502 07:58:33.327746 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240502 07:58:33.338882 21568 heartbeater.cc:347] Connected to a master server at 127.15.129.190:41083
I20240502 07:58:33.339099 21568 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:33.339536 21568 heartbeater.cc:511] Master 127.15.129.190:41083 requested a full tablet report, sending...
I20240502 07:58:33.340474 17728 ts_manager.cc:194] Registered new tserver with Master: a0d44a6c162042df9028884a953767ff (127.15.129.136:33243)
I20240502 07:58:33.341241 17728 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.136:34281
I20240502 07:58:33.533691 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:33.542416 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:33.543205 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:33.545059 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:33.545580 21502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:33.551324 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:33.559829 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:33.574620 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:33.686269 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:33.688153 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:33.688716 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:33.691718 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:33.694645 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:33.703186 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:33.710235 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:33.724303 21502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:33.765846 17873 consensus_queue.cc:237] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } }
W20240502 07:58:33.771288 17808 consensus_peers.cc:485] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa -> Peer a0d44a6c162042df9028884a953767ff (127.15.129.136:33243): Couldn't send request to peer a0d44a6c162042df9028884a953767ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 334047bb4be641e0aad4f6405e067468. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:34.343269 21568 heartbeater.cc:503] Master 127.15.129.190:41083 was elected leader, sending a full tablet report...
I20240502 07:58:34.386602 21674 ts_tablet_manager.cc:902] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff: Initiating tablet copy from peer 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343)
I20240502 07:58:34.387310 21674 tablet_copy_client.cc:308] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.129:42343
I20240502 07:58:34.391024 17894 tablet_copy_service.cc:140] P 3d36406429504d36ab670d84f4464efa: Received BeginTabletCopySession request for tablet 334047bb4be641e0aad4f6405e067468 from peer a0d44a6c162042df9028884a953767ff ({username='slave'} at 127.15.129.136:54151)
I20240502 07:58:34.391144 17894 tablet_copy_service.cc:161] P 3d36406429504d36ab670d84f4464efa: Beginning new tablet copy session on tablet 334047bb4be641e0aad4f6405e067468 from peer a0d44a6c162042df9028884a953767ff at {username='slave'} at 127.15.129.136:54151: session id = a0d44a6c162042df9028884a953767ff-334047bb4be641e0aad4f6405e067468
I20240502 07:58:34.391911 17894 tablet_copy_source_session.cc:205] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:34.395007 21674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 334047bb4be641e0aad4f6405e067468. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:34.399909 21674 tablet_copy_client.cc:783] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:34.400171 21674 tablet_copy_client.cc:647] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:34.402845 21674 tablet_copy_client.cc:523] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:34.404467 21674 tablet_bootstrap.cc:492] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff: Bootstrap starting.
I20240502 07:58:34.409757 21674 log.cc:824] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:34.429309 21674 tablet_bootstrap.cc:492] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 07:58:34.429750 21674 tablet_bootstrap.cc:492] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff: Bootstrap complete.
I20240502 07:58:34.430119 21674 ts_tablet_manager.cc:1367] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff: Time spent bootstrapping tablet: real 0.026s user 0.022s sys 0.004s
I20240502 07:58:34.432093 21674 raft_consensus.cc:348] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } }
I20240502 07:58:34.432629 21674 raft_consensus.cc:729] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0d44a6c162042df9028884a953767ff, State: Initialized, Role: FOLLOWER
I20240502 07:58:34.432915 21674 consensus_queue.cc:260] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } }
I20240502 07:58:34.433693 21674 ts_tablet_manager.cc:1398] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20240502 07:58:34.434435 17894 tablet_copy_service.cc:342] P 3d36406429504d36ab670d84f4464efa: Request end of tablet copy session a0d44a6c162042df9028884a953767ff-334047bb4be641e0aad4f6405e067468 received from {username='slave'} at 127.15.129.136:54151
I20240502 07:58:34.434535 17894 tablet_copy_service.cc:433] P 3d36406429504d36ab670d84f4464efa: ending tablet copy session a0d44a6c162042df9028884a953767ff-334047bb4be641e0aad4f6405e067468 on tablet 334047bb4be641e0aad4f6405e067468 with peer a0d44a6c162042df9028884a953767ff
I20240502 07:58:34.781714 21522 raft_consensus.cc:1203] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:34.782920 21683 raft_consensus.cc:2932] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a0d44a6c162042df9028884a953767ff (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } } }
I20240502 07:58:34.783591 21522 raft_consensus.cc:2932] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a0d44a6c162042df9028884a953767ff (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } } }
I20240502 07:58:34.784482 21683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:47182 (request call id 0) took 1019 ms. Trace:
I20240502 07:58:34.784577 21683 rpcz_store.cc:276] 0502 07:58:33.765489 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 07:58:33.765553 (+ 64us) service_pool.cc:225] Handling call
0502 07:58:33.766329 (+ 776us) log.cc:842] Serialized 263 byte log entry
0502 07:58:34.784464 (+1018135us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":73,"raft.queue_time_us":113,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"thread_start_us":104,"threads_started":2,"wal-append.queue_time_us":125,"wal-append.run_cpu_time_us":299,"wal-append.run_wall_time_us":1000367}
I20240502 07:58:34.785962 17729 catalog_manager.cc:5484] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa reported cstate change: config changed from index -1 to 4, VOTER a0d44a6c162042df9028884a953767ff (127.15.129.136) added. New cstate: current_term: 1 leader_uuid: "3d36406429504d36ab670d84f4464efa" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } } }
I20240502 07:58:34.805239 18005 consensus_queue.cc:237] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } }
W20240502 07:58:34.810305 17939 consensus_peers.cc:485] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 -> Peer a0d44a6c162042df9028884a953767ff (127.15.129.136:33243): Couldn't send request to peer a0d44a6c162042df9028884a953767ff. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 04638eb3c88040f7b40f7f6ec6989e06. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:35.421737 21713 ts_tablet_manager.cc:902] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff: Initiating tablet copy from peer 923d7729977f4edda0aa760e184a8830 (127.15.129.130:45189)
I20240502 07:58:35.422191 21713 tablet_copy_client.cc:308] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.130:45189
I20240502 07:58:35.425484 18025 tablet_copy_service.cc:140] P 923d7729977f4edda0aa760e184a8830: Received BeginTabletCopySession request for tablet 04638eb3c88040f7b40f7f6ec6989e06 from peer a0d44a6c162042df9028884a953767ff ({username='slave'} at 127.15.129.136:55701)
I20240502 07:58:35.425576 18025 tablet_copy_service.cc:161] P 923d7729977f4edda0aa760e184a8830: Beginning new tablet copy session on tablet 04638eb3c88040f7b40f7f6ec6989e06 from peer a0d44a6c162042df9028884a953767ff at {username='slave'} at 127.15.129.136:55701: session id = a0d44a6c162042df9028884a953767ff-04638eb3c88040f7b40f7f6ec6989e06
I20240502 07:58:35.426254 18025 tablet_copy_source_session.cc:205] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:35.427100 21713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04638eb3c88040f7b40f7f6ec6989e06. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:35.428725 21713 tablet_copy_client.cc:783] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:35.428859 21713 tablet_copy_client.cc:647] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:35.430032 21713 tablet_copy_client.cc:523] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:35.430950 21713 tablet_bootstrap.cc:492] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff: Bootstrap starting.
I20240502 07:58:35.447567 21713 tablet_bootstrap.cc:492] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240502 07:58:35.447849 21713 tablet_bootstrap.cc:492] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff: Bootstrap complete.
I20240502 07:58:35.447999 21713 ts_tablet_manager.cc:1367] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff: Time spent bootstrapping tablet: real 0.017s user 0.015s sys 0.000s
I20240502 07:58:35.448470 21713 raft_consensus.cc:348] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } }
I20240502 07:58:35.448670 21713 raft_consensus.cc:729] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0d44a6c162042df9028884a953767ff, State: Initialized, Role: FOLLOWER
I20240502 07:58:35.448777 21713 consensus_queue.cc:260] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } }
I20240502 07:58:35.449087 21713 ts_tablet_manager.cc:1398] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 07:58:35.449720 18025 tablet_copy_service.cc:342] P 923d7729977f4edda0aa760e184a8830: Request end of tablet copy session a0d44a6c162042df9028884a953767ff-04638eb3c88040f7b40f7f6ec6989e06 received from {username='slave'} at 127.15.129.136:55701
I20240502 07:58:35.449801 18025 tablet_copy_service.cc:433] P 923d7729977f4edda0aa760e184a8830: ending tablet copy session a0d44a6c162042df9028884a953767ff-04638eb3c88040f7b40f7f6ec6989e06 on tablet 04638eb3c88040f7b40f7f6ec6989e06 with peer a0d44a6c162042df9028884a953767ff
I20240502 07:58:35.907603 21522 raft_consensus.cc:1203] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:35.908608 21722 raft_consensus.cc:2932] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a0d44a6c162042df9028884a953767ff (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } } }
I20240502 07:58:35.909006 21522 raft_consensus.cc:2932] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER a0d44a6c162042df9028884a953767ff (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } } }
I20240502 07:58:35.910130 21722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:47536 (request call id 0) took 1105 ms. Trace:
I20240502 07:58:35.910241 21722 rpcz_store.cc:276] 0502 07:58:34.804872 (+ 0us) service_pool.cc:168] Inserting onto call queue
0502 07:58:34.804938 (+ 66us) service_pool.cc:225] Handling call
0502 07:58:34.805807 (+ 869us) log.cc:842] Serialized 263 byte log entry
0502 07:58:35.910112 (+1104305us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":47,"raft.queue_time_us":112,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":148,"spinlock_wait_cycles":13568,"thread_start_us":122,"threads_started":2,"wal-append.queue_time_us":167,"wal-append.run_cpu_time_us":359,"wal-append.run_wall_time_us":1000462}
I20240502 07:58:35.911450 17732 catalog_manager.cc:5484] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 reported cstate change: config changed from index -1 to 4, VOTER a0d44a6c162042df9028884a953767ff (127.15.129.136) added. New cstate: current_term: 1 leader_uuid: "923d7729977f4edda0aa760e184a8830" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } } }
I20240502 07:58:35.969867 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:35.970484 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:35.970515 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:35.972018 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:35.978667 21502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:35.996237 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:35.998131 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.005811 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.030032 18005 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "04638eb3c88040f7b40f7f6ec6989e06"
dest_uuid: "923d7729977f4edda0aa760e184a8830"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:47556
I20240502 07:58:36.030179 18005 raft_consensus.cc:595] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:58:36.082664 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:36.082791 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.083099 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:36.085395 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.091185 21502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:36.095461 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.115042 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.122341 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.197749 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:36.202592 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.203400 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:36.214613 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.214767 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.217285 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.221678 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.223065 21502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:36.255105 17873 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "334047bb4be641e0aad4f6405e067468"
dest_uuid: "3d36406429504d36ab670d84f4464efa"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:57784
I20240502 07:58:36.255250 17873 raft_consensus.cc:595] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Received request to transfer leadership
I20240502 07:58:36.309013 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.309491 17984 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:36.310066 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:36.318620 21502 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:36.322587 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.322839 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.333827 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.345957 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.369976 21712 raft_consensus.cc:979] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830: : Instructing follower a0d44a6c162042df9028884a953767ff to start an election
I20240502 07:58:36.370100 21722 raft_consensus.cc:1067] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Signalling peer a0d44a6c162042df9028884a953767ff to start an election
I20240502 07:58:36.370556 21522 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "04638eb3c88040f7b40f7f6ec6989e06"
dest_uuid: "a0d44a6c162042df9028884a953767ff"
from {username='slave'} at 127.15.129.130:54237
I20240502 07:58:36.370764 21522 raft_consensus.cc:482] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:58:36.370874 21522 raft_consensus.cc:3037] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:36.372020 21522 raft_consensus.cc:504] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } }
I20240502 07:58:36.372887 21522 leader_election.cc:288] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff [CANDIDATE]: Term 2 election: Requested vote from peers 923d7729977f4edda0aa760e184a8830 (127.15.129.130:45189)
I20240502 07:58:36.373379 18005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04638eb3c88040f7b40f7f6ec6989e06" candidate_uuid: "a0d44a6c162042df9028884a953767ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "923d7729977f4edda0aa760e184a8830"
I20240502 07:58:36.373497 18005 raft_consensus.cc:3032] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:58:36.373548 18005 raft_consensus.cc:729] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 923d7729977f4edda0aa760e184a8830, State: Running, Role: LEADER
I20240502 07:58:36.373703 18005 consensus_queue.cc:260] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } }
I20240502 07:58:36.373838 18005 raft_consensus.cc:3037] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:36.374656 18005 raft_consensus.cc:2454] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0d44a6c162042df9028884a953767ff in term 2.
I20240502 07:58:36.375044 21457 leader_election.cc:302] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 923d7729977f4edda0aa760e184a8830, a0d44a6c162042df9028884a953767ff; no voters:
I20240502 07:58:36.375408 21721 raft_consensus.cc:2781] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:58:36.375556 21721 raft_consensus.cc:686] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff [term 2 LEADER]: Becoming Leader. State: Replica: a0d44a6c162042df9028884a953767ff, State: Running, Role: LEADER
I20240502 07:58:36.375834 21721 consensus_queue.cc:237] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } }
I20240502 07:58:36.377672 17731 catalog_manager.cc:5484] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff reported cstate change: term changed from 1 to 2, leader changed from 923d7729977f4edda0aa760e184a8830 (127.15.129.130) to a0d44a6c162042df9028884a953767ff (127.15.129.136). New cstate: current_term: 2 leader_uuid: "a0d44a6c162042df9028884a953767ff" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 } } peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } } }
I20240502 07:58:36.638612 21502 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:36.638942 17853 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:36.639153 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.640363 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.641587 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.658181 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.659663 17984 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.663386 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.680958 21928 raft_consensus.cc:979] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa: : Instructing follower a0d44a6c162042df9028884a953767ff to start an election
I20240502 07:58:36.681067 21928 raft_consensus.cc:1067] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Signalling peer a0d44a6c162042df9028884a953767ff to start an election
I20240502 07:58:36.681507 21522 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "334047bb4be641e0aad4f6405e067468"
dest_uuid: "a0d44a6c162042df9028884a953767ff"
from {username='slave'} at 127.15.129.129:36615
I20240502 07:58:36.681661 21522 raft_consensus.cc:482] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:58:36.681720 21522 raft_consensus.cc:3037] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:36.682714 21522 raft_consensus.cc:504] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } }
I20240502 07:58:36.683071 21522 leader_election.cc:288] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff [CANDIDATE]: Term 2 election: Requested vote from peers 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343)
I20240502 07:58:36.683432 17873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "334047bb4be641e0aad4f6405e067468" candidate_uuid: "a0d44a6c162042df9028884a953767ff" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d36406429504d36ab670d84f4464efa"
I20240502 07:58:36.683576 17873 raft_consensus.cc:3032] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:58:36.683624 17873 raft_consensus.cc:729] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d36406429504d36ab670d84f4464efa, State: Running, Role: LEADER
I20240502 07:58:36.683765 17873 consensus_queue.cc:260] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } }
I20240502 07:58:36.683903 17873 raft_consensus.cc:3037] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:58:36.684836 17873 raft_consensus.cc:2454] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0d44a6c162042df9028884a953767ff in term 2.
I20240502 07:58:36.685137 21455 leader_election.cc:302] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 3d36406429504d36ab670d84f4464efa, a0d44a6c162042df9028884a953767ff; no voters:
I20240502 07:58:36.685295 21889 raft_consensus.cc:2781] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:58:36.685410 21889 raft_consensus.cc:686] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff [term 2 LEADER]: Becoming Leader. State: Replica: a0d44a6c162042df9028884a953767ff, State: Running, Role: LEADER
I20240502 07:58:36.685547 21889 consensus_queue.cc:237] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } }
I20240502 07:58:36.687111 17732 catalog_manager.cc:5484] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff reported cstate change: term changed from 1 to 2, leader changed from 3d36406429504d36ab670d84f4464efa (127.15.129.129) to a0d44a6c162042df9028884a953767ff (127.15.129.136). New cstate: current_term: 2 leader_uuid: "a0d44a6c162042df9028884a953767ff" committed_config { opid_index: 4 OBSOLETE_local: true peers { permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 } } peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } } }
I20240502 07:58:36.752472 18005 raft_consensus.cc:1261] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Refusing update from remote peer a0d44a6c162042df9028884a953767ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:58:36.753250 21889 consensus_queue.cc:1035] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "923d7729977f4edda0aa760e184a8830" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 45189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:36.759017 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.760965 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.761933 17853 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.766204 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.775156 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.775578 17984 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.792820 21502 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:36.797982 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.086882 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.090727 17853 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.095274 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.095479 17984 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.103255 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.104736 21502 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.111001 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.136255 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.172504 21522 consensus_queue.cc:237] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } }
I20240502 07:58:37.173220 21721 raft_consensus.cc:2932] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 923d7729977f4edda0aa760e184a8830 (127.15.129.130) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } } }
I20240502 07:58:37.175987 17730 catalog_manager.cc:5484] T 04638eb3c88040f7b40f7f6ec6989e06 P a0d44a6c162042df9028884a953767ff reported cstate change: config changed from index 4 to 6, VOTER 923d7729977f4edda0aa760e184a8830 (127.15.129.130) evicted. New cstate: current_term: 2 leader_uuid: "a0d44a6c162042df9028884a953767ff" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } } }
I20240502 07:58:37.178761 17984 tablet_service.cc:1514] Processing DeleteTablet for tablet 04638eb3c88040f7b40f7f6ec6989e06 with delete_type TABLET_DATA_TOMBSTONED (TS 923d7729977f4edda0aa760e184a8830 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45534
I20240502 07:58:37.179154 22029 tablet_replica.cc:330] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830: stopping tablet replica
I20240502 07:58:37.179761 22029 raft_consensus.cc:2229] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:37.179896 22029 raft_consensus.cc:2258] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:37.181653 22029 ts_tablet_manager.cc:1875] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:37.183974 22029 ts_tablet_manager.cc:1888] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 07:58:37.184230 22029 log.cc:1196] T 04638eb3c88040f7b40f7f6ec6989e06 P 923d7729977f4edda0aa760e184a8830: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/04638eb3c88040f7b40f7f6ec6989e06
I20240502 07:58:37.184986 17718 catalog_manager.cc:4830] TS 923d7729977f4edda0aa760e184a8830 (127.15.129.130:45189): tablet 04638eb3c88040f7b40f7f6ec6989e06 (table rebalance_test_table_4 [id=aa24b74e8feb4d10b47ce377487513ca]) successfully deleted
I20240502 07:58:37.231968 17873 raft_consensus.cc:1261] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Refusing update from remote peer a0d44a6c162042df9028884a953767ff: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:58:37.233201 21721 consensus_queue.cc:1035] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d36406429504d36ab670d84f4464efa" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 42343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:37.237237 17853 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.237888 21502 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.245404 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.245711 17984 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.248265 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.249089 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.260190 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.261857 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.310928 21522 consensus_queue.cc:237] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } }
I20240502 07:58:37.311496 21889 raft_consensus.cc:2932] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER 3d36406429504d36ab670d84f4464efa (127.15.129.129) evicted. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } } }
I20240502 07:58:37.313673 17732 catalog_manager.cc:5484] T 334047bb4be641e0aad4f6405e067468 P a0d44a6c162042df9028884a953767ff reported cstate change: config changed from index 4 to 6, VOTER 3d36406429504d36ab670d84f4464efa (127.15.129.129) evicted. New cstate: current_term: 2 leader_uuid: "a0d44a6c162042df9028884a953767ff" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a0d44a6c162042df9028884a953767ff" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 33243 } } }
I20240502 07:58:37.316179 17853 tablet_service.cc:1514] Processing DeleteTablet for tablet 334047bb4be641e0aad4f6405e067468 with delete_type TABLET_DATA_TOMBSTONED (TS 3d36406429504d36ab670d84f4464efa not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:50568
I20240502 07:58:37.316535 22082 tablet_replica.cc:330] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa: stopping tablet replica
I20240502 07:58:37.316665 22082 raft_consensus.cc:2229] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:37.316783 22082 raft_consensus.cc:2258] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:37.318369 22082 ts_tablet_manager.cc:1875] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:37.320461 22082 ts_tablet_manager.cc:1888] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 07:58:37.320531 22082 log.cc:1196] T 334047bb4be641e0aad4f6405e067468 P 3d36406429504d36ab670d84f4464efa: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/334047bb4be641e0aad4f6405e067468
I20240502 07:58:37.321240 17716 catalog_manager.cc:4830] TS 3d36406429504d36ab670d84f4464efa (127.15.129.129:42343): tablet 334047bb4be641e0aad4f6405e067468 (table rebalance_test_table_2 [id=46e1dc9f151748c996c37174bb27644c]) successfully deleted
I20240502 07:58:37.583007 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.587555 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.590265 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.594825 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.599726 17853 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.600121 17984 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.617657 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.618559 21502 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.699381 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.701225 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.702323 17853 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.702577 17984 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.707834 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.719923 21502 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.720438 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.741788 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.828387 17853 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.830052 19123 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.837752 19395 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.839318 17984 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.845724 21502 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.846382 18992 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.847456 19526 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:37.861531 19255 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
53be75cf33b74f38884080e1936c65f1 | 127.15.129.190:41083 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
allow_unsafe_replication_factor | true | runtime,unsafe | all 1 server(s) checked
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+----------------------+-------------------------
builtin_ntp_servers | 127.15.129.148:41341 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
3d36406429504d36ab670d84f4464efa | 127.15.129.129:42343 | HEALTHY | <none> | 2 | 0
3f7795cc262b43cfa8a9252c424b7354 | 127.15.129.132:43973 | HEALTHY | <none> | 2 | 0
451db10510a9469b92abc0705d7bf44d | 127.15.129.133:34385 | HEALTHY | <none> | 2 | 0
923d7729977f4edda0aa760e184a8830 | 127.15.129.130:45189 | HEALTHY | <none> | 2 | 0
9fdc864a7d9349808a6fc7b2cf4eba8f | 127.15.129.131:43267 | HEALTHY | <none> | 2 | 0
a0d44a6c162042df9028884a953767ff | 127.15.129.136:33243 | HEALTHY | <none> | 2 | 0
c815aa98415f4682b2c8429a67e85444 | 127.15.129.134:34275 | HEALTHY | <none> | 2 | 0
d87eea2501954fca83327b9408429b66 | 127.15.129.135:37247 | HEALTHY | <none> | 2 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 8
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+-------------------------
ipki_server_key_size | 768 | experimental | all 8 server(s) checked
local_ip_for_outbound_sockets | 127.15.129.129 | experimental | 127.15.129.129:42343
local_ip_for_outbound_sockets | 127.15.129.130 | experimental | 127.15.129.130:45189
local_ip_for_outbound_sockets | 127.15.129.131 | experimental | 127.15.129.131:43267
local_ip_for_outbound_sockets | 127.15.129.132 | experimental | 127.15.129.132:43973
local_ip_for_outbound_sockets | 127.15.129.133 | experimental | 127.15.129.133:34385
local_ip_for_outbound_sockets | 127.15.129.134 | experimental | 127.15.129.134:34275
local_ip_for_outbound_sockets | 127.15.129.135 | experimental | 127.15.129.135:37247
local_ip_for_outbound_sockets | 127.15.129.136 | experimental | 127.15.129.136:33243
never_fsync | true | unsafe,advanced | all 8 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 8 server(s) checked
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 8 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 8 server(s) checked
server_dump_info_format | pb | hidden | all 8 server(s) checked
server_dump_info_path | /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.15.129.129:42343
server_dump_info_path | /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.15.129.130:45189
server_dump_info_path | /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.15.129.131:43267
server_dump_info_path | /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden | 127.15.129.132:43973
server_dump_info_path | /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden | 127.15.129.133:34385
server_dump_info_path | /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden | 127.15.129.134:34275
server_dump_info_path | /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden | 127.15.129.135:37247
server_dump_info_path | /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalanceParamTest.Rebalance_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/info.pb | hidden | 127.15.129.136:33243
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+----------------------+-------------------------
builtin_ntp_servers | 127.15.129.148:41341 | all 8 server(s) checked
time_source | builtin | all 8 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 9 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
rebalance_test_table_0 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_1 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_2 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_3 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
rebalance_test_table_4 | 1 | HEALTHY | 3 | 3 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 2
First Quartile | 2
Median | 2
Third Quartile | 2
Maximum | 2
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 8
Tables | 6
Tablets | 16
Replicas | 16
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20240502 07:58:37.890551 15878 log_verifier.cc:126] Checking tablet 04638eb3c88040f7b40f7f6ec6989e06
I20240502 07:58:37.900054 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 04638eb3c88040f7b40f7f6ec6989e06
I20240502 07:58:37.900120 15878 log_verifier.cc:126] Checking tablet 1042d2331b5a425eb8e6721cb5c4b894
I20240502 07:58:37.905730 15878 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 1042d2331b5a425eb8e6721cb5c4b894
I20240502 07:58:37.905797 15878 log_verifier.cc:126] Checking tablet 323b33d450384c90b13a1ac565f411a4
I20240502 07:58:37.911875 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 323b33d450384c90b13a1ac565f411a4
I20240502 07:58:37.911944 15878 log_verifier.cc:126] Checking tablet 334047bb4be641e0aad4f6405e067468
I20240502 07:58:37.920250 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 334047bb4be641e0aad4f6405e067468
I20240502 07:58:37.920321 15878 log_verifier.cc:126] Checking tablet 3f6c4761aef54c538693f1d3a4d80d10
I20240502 07:58:37.925987 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 3f6c4761aef54c538693f1d3a4d80d10
I20240502 07:58:37.926049 15878 log_verifier.cc:126] Checking tablet 41a4e8bf2f494f159b25a8437f7bdeec
I20240502 07:58:37.930730 15878 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 41a4e8bf2f494f159b25a8437f7bdeec
I20240502 07:58:37.930794 15878 log_verifier.cc:126] Checking tablet 4609595bd09d4893b2575ca6a2740896
I20240502 07:58:37.936558 15878 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 4609595bd09d4893b2575ca6a2740896
I20240502 07:58:37.936623 15878 log_verifier.cc:126] Checking tablet 4ae9214f8e5a4113b339c78cf80b1ed8
I20240502 07:58:37.942822 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 4ae9214f8e5a4113b339c78cf80b1ed8
I20240502 07:58:37.942883 15878 log_verifier.cc:126] Checking tablet 4fb3087aff7d40eb960135010402371e
I20240502 07:58:37.948482 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 4fb3087aff7d40eb960135010402371e
I20240502 07:58:37.948561 15878 log_verifier.cc:126] Checking tablet 62ba9d55df884c568ea6285701011ceb
I20240502 07:58:37.955017 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet 62ba9d55df884c568ea6285701011ceb
I20240502 07:58:37.955082 15878 log_verifier.cc:126] Checking tablet 706476e790b54b2690ea892fb6850046
I20240502 07:58:37.960558 15878 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 706476e790b54b2690ea892fb6850046
I20240502 07:58:37.960624 15878 log_verifier.cc:126] Checking tablet 9d165c26beff4f3dacf61ca92fdec35f
I20240502 07:58:37.966192 15878 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 9d165c26beff4f3dacf61ca92fdec35f
I20240502 07:58:37.966254 15878 log_verifier.cc:126] Checking tablet a7a2522085124c63a938200b3e925cdb
I20240502 07:58:37.971920 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet a7a2522085124c63a938200b3e925cdb
I20240502 07:58:37.971972 15878 log_verifier.cc:126] Checking tablet bab58d3c92774b1ab9fa77e100db9073
I20240502 07:58:37.977775 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet bab58d3c92774b1ab9fa77e100db9073
I20240502 07:58:37.977852 15878 log_verifier.cc:126] Checking tablet e82159be5de549fe9ed93db5ef6262fc
I20240502 07:58:37.983510 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet e82159be5de549fe9ed93db5ef6262fc
I20240502 07:58:37.983566 15878 log_verifier.cc:126] Checking tablet f49ef9b8bef141a3bc00ec7bf7c1ac67
I20240502 07:58:37.989216 15878 log_verifier.cc:177] Verified matching terms for 6 ops in tablet f49ef9b8bef141a3bc00ec7bf7c1ac67
I20240502 07:58:37.989703 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 17792
I20240502 07:58:37.996949 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 17923
I20240502 07:58:38.004482 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 18930
I20240502 07:58:38.011265 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 19061
I20240502 07:58:38.016256 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 19192
I20240502 07:58:38.022266 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 19325
I20240502 07:58:38.028244 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 19464
I20240502 07:58:38.034618 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 21440
I20240502 07:58:38.040844 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 17701
2024-05-02T07:58:38Z chronyd exiting
[ OK ] RebalanceParamTest.Rebalance/0 (27386 ms)
[----------] 1 test from RebalanceParamTest (27386 ms total)
[----------] 1 test from DDLDuringRebalancingTest
[ RUN ] DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing/0
I20240502 07:58:38.080967 15878 test_util.cc:268] Using random seed: -125828159
I20240502 07:58:38.081802 15878 ts_itest-base.cc:112] Starting cluster with:
I20240502 07:58:38.081845 15878 ts_itest-base.cc:113] --------------
I20240502 07:58:38.081862 15878 ts_itest-base.cc:114] 8 tablet servers
I20240502 07:58:38.081878 15878 ts_itest-base.cc:115] 3 replicas per TS
I20240502 07:58:38.081895 15878 ts_itest-base.cc:116] --------------
2024-05-02T07:58:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T07:58:38Z Disabled control of system clock
I20240502 07:58:38.090842 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.129.190:35417
--webserver_interface=127.15.129.190
--webserver_port=0
--builtin_ntp_servers=127.15.129.148:39703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.129.190:35417
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=false
--enable_range_replica_placement=false with env {}
W20240502 07:58:38.202566 22199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:38.202807 22199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:38.202848 22199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:38.205876 22199 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:38.205941 22199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 07:58:38.205965 22199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:38.205983 22199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 07:58:38.206000 22199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 07:58:38.209686 22199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.15.129.190:35417
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.129.190:35417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.15.129.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.22199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:38.210583 22199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:38.211707 22199 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:38.220077 22205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:38.221443 22204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:38.222424 22207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:38.223520 22199 server_base.cc:1034] running on GCE node
I20240502 07:58:38.224067 22199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:38.225044 22199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:38.226219 22199 hybrid_clock.cc:648] HybridClock initialized: now 1714636718226206 us; error 35 us; skew 500 ppm
I20240502 07:58:38.228199 22199 webserver.cc:460] Webserver started at http://127.15.129.190:38235/ using document root <none> and password file <none>
I20240502 07:58:38.228768 22199 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:38.228838 22199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:38.229014 22199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:38.230734 22199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "f9724f21fce0432bad8bbeb45d297445"
format_stamp: "Formatted at 2024-05-02 07:58:38 on dist-test-slave-0hx2"
I20240502 07:58:38.231215 22199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "f9724f21fce0432bad8bbeb45d297445"
format_stamp: "Formatted at 2024-05-02 07:58:38 on dist-test-slave-0hx2"
I20240502 07:58:38.234858 22199 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 07:58:38.237171 22213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:38.238298 22199 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:58:38.238425 22199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
uuid: "f9724f21fce0432bad8bbeb45d297445"
format_stamp: "Formatted at 2024-05-02 07:58:38 on dist-test-slave-0hx2"
I20240502 07:58:38.238523 22199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:38.253077 22199 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:38.253759 22199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:38.253921 22199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:38.260607 22199 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.190:35417
I20240502 07:58:38.260625 22265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.190:35417 every 8 connection(s)
I20240502 07:58:38.261636 22199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 07:58:38.264186 22266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:38.266572 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 22199
I20240502 07:58:38.266700 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 07:58:38.269629 22266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Bootstrap starting.
I20240502 07:58:38.271936 22266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:38.272769 22266 log.cc:824] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:38.274456 22266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: No bootstrap required, opened a new log
I20240502 07:58:38.277249 22266 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9724f21fce0432bad8bbeb45d297445" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 35417 } }
I20240502 07:58:38.277480 22266 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:38.277549 22266 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9724f21fce0432bad8bbeb45d297445, State: Initialized, Role: FOLLOWER
I20240502 07:58:38.277978 22266 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9724f21fce0432bad8bbeb45d297445" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 35417 } }
I20240502 07:58:38.278106 22266 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:58:38.278173 22266 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:58:38.278283 22266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:38.279438 22266 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9724f21fce0432bad8bbeb45d297445" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 35417 } }
I20240502 07:58:38.279768 22266 leader_election.cc:302] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9724f21fce0432bad8bbeb45d297445; no voters:
I20240502 07:58:38.280052 22266 leader_election.cc:288] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:58:38.280408 22271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:38.280606 22271 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445 [term 1 LEADER]: Becoming Leader. State: Replica: f9724f21fce0432bad8bbeb45d297445, State: Running, Role: LEADER
I20240502 07:58:38.280898 22271 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9724f21fce0432bad8bbeb45d297445" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 35417 } }
I20240502 07:58:38.281174 22266 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 07:58:38.282598 22272 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9724f21fce0432bad8bbeb45d297445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9724f21fce0432bad8bbeb45d297445" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 35417 } } }
I20240502 07:58:38.282635 22273 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9724f21fce0432bad8bbeb45d297445. Latest consensus state: current_term: 1 leader_uuid: "f9724f21fce0432bad8bbeb45d297445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9724f21fce0432bad8bbeb45d297445" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 35417 } } }
I20240502 07:58:38.282721 22272 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445 [sys.catalog]: This master's current role is: LEADER
I20240502 07:58:38.282784 22273 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445 [sys.catalog]: This master's current role is: LEADER
I20240502 07:58:38.283093 22280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 07:58:38.286096 22280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 07:58:38.291944 22280 catalog_manager.cc:1347] Generated new cluster ID: 1e52df0abaa74bc7bc611aebb6dfc060
I20240502 07:58:38.292023 22280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 07:58:38.301951 22280 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 07:58:38.302911 22280 catalog_manager.cc:1504] Loading token signing keys...
I20240502 07:58:38.310309 22280 catalog_manager.cc:5857] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Generated new TSK 0
I20240502 07:58:38.311093 22280 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 07:58:38.325697 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.129:0
--local_ip_for_outbound_sockets=127.15.129.129
--webserver_interface=127.15.129.129
--webserver_port=0
--tserver_master_addrs=127.15.129.190:35417
--builtin_ntp_servers=127.15.129.148:39703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 07:58:38.436466 22290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:38.436729 22290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:38.436808 22290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:38.440249 22290 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:38.440361 22290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:38.440479 22290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.129
I20240502 07:58:38.444532 22290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.15.129.129
--webserver_port=0
--tserver_master_addrs=127.15.129.190:35417
--never_fsync=true
--heap_profile_path=/tmp/kudu.22290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.129
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:38.445549 22290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:38.446692 22290 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:38.456302 22295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:38.457120 22298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:38.458566 22296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:38.459757 22290 server_base.cc:1034] running on GCE node
I20240502 07:58:38.460134 22290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:38.460690 22290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:38.461853 22290 hybrid_clock.cc:648] HybridClock initialized: now 1714636718461838 us; error 31 us; skew 500 ppm
I20240502 07:58:38.463873 22290 webserver.cc:460] Webserver started at http://127.15.129.129:36199/ using document root <none> and password file <none>
I20240502 07:58:38.464387 22290 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:38.464440 22290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:38.464597 22290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:38.466284 22290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "b533a7d8d97943609c6e28e3e36b78a4"
format_stamp: "Formatted at 2024-05-02 07:58:38 on dist-test-slave-0hx2"
I20240502 07:58:38.466732 22290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "b533a7d8d97943609c6e28e3e36b78a4"
format_stamp: "Formatted at 2024-05-02 07:58:38 on dist-test-slave-0hx2"
I20240502 07:58:38.470507 22290 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20240502 07:58:38.472904 22304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:38.474083 22290 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:58:38.474201 22290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "b533a7d8d97943609c6e28e3e36b78a4"
format_stamp: "Formatted at 2024-05-02 07:58:38 on dist-test-slave-0hx2"
I20240502 07:58:38.474305 22290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:38.493228 22290 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:38.493957 22290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:38.494130 22290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:38.494695 22290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:38.495582 22290 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:38.495630 22290 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:38.495685 22290 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:38.495719 22290 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:38.504212 22290 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.129:38535
I20240502 07:58:38.504292 22417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.129:38535 every 8 connection(s)
I20240502 07:58:38.505185 22290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 07:58:38.512128 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 22290
I20240502 07:58:38.512275 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 07:58:38.514357 22418 heartbeater.cc:347] Connected to a master server at 127.15.129.190:35417
I20240502 07:58:38.514611 22418 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:38.514716 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.130:0
--local_ip_for_outbound_sockets=127.15.129.130
--webserver_interface=127.15.129.130
--webserver_port=0
--tserver_master_addrs=127.15.129.190:35417
--builtin_ntp_servers=127.15.129.148:39703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:38.515170 22418 heartbeater.cc:511] Master 127.15.129.190:35417 requested a full tablet report, sending...
I20240502 07:58:38.516887 22229 ts_manager.cc:194] Registered new tserver with Master: b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:38.518092 22229 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.129:54999
W20240502 07:58:38.625886 22421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:38.626128 22421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:38.626206 22421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:38.629221 22421 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:38.629297 22421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:38.629423 22421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.130
I20240502 07:58:38.633092 22421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.15.129.130
--webserver_port=0
--tserver_master_addrs=127.15.129.190:35417
--never_fsync=true
--heap_profile_path=/tmp/kudu.22421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.130
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:38.634052 22421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:38.635176 22421 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:38.644457 22429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:38.645363 22426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:38.646328 22427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:38.647887 22421 server_base.cc:1034] running on GCE node
I20240502 07:58:38.648315 22421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:38.648905 22421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:38.650092 22421 hybrid_clock.cc:648] HybridClock initialized: now 1714636718650064 us; error 40 us; skew 500 ppm
I20240502 07:58:38.655930 22421 webserver.cc:460] Webserver started at http://127.15.129.130:36245/ using document root <none> and password file <none>
I20240502 07:58:38.656527 22421 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:38.656595 22421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:38.656781 22421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:38.658497 22421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "fabddb16c85944648a710ec52fc7244c"
format_stamp: "Formatted at 2024-05-02 07:58:38 on dist-test-slave-0hx2"
I20240502 07:58:38.658964 22421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "fabddb16c85944648a710ec52fc7244c"
format_stamp: "Formatted at 2024-05-02 07:58:38 on dist-test-slave-0hx2"
I20240502 07:58:38.662253 22421 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:58:38.664418 22435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:38.665436 22421 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:58:38.665549 22421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "fabddb16c85944648a710ec52fc7244c"
format_stamp: "Formatted at 2024-05-02 07:58:38 on dist-test-slave-0hx2"
I20240502 07:58:38.665637 22421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:38.701059 22421 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:38.701792 22421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:38.701959 22421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:38.702519 22421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:38.703413 22421 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:38.703469 22421 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:38.703534 22421 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:38.703560 22421 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:38.712066 22421 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.130:39625
I20240502 07:58:38.712092 22548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.130:39625 every 8 connection(s)
I20240502 07:58:38.713229 22421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 07:58:38.721663 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 22421
I20240502 07:58:38.721791 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 07:58:38.722275 22549 heartbeater.cc:347] Connected to a master server at 127.15.129.190:35417
I20240502 07:58:38.722520 22549 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:38.723057 22549 heartbeater.cc:511] Master 127.15.129.190:35417 requested a full tablet report, sending...
I20240502 07:58:38.724081 22229 ts_manager.cc:194] Registered new tserver with Master: fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:38.724113 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.131:0
--local_ip_for_outbound_sockets=127.15.129.131
--webserver_interface=127.15.129.131
--webserver_port=0
--tserver_master_addrs=127.15.129.190:35417
--builtin_ntp_servers=127.15.129.148:39703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:38.724622 22229 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.130:38889
W20240502 07:58:38.829692 22552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:38.829926 22552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:38.829998 22552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:38.833012 22552 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:38.833093 22552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:38.833196 22552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.131
I20240502 07:58:38.836772 22552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.15.129.131
--webserver_port=0
--tserver_master_addrs=127.15.129.190:35417
--never_fsync=true
--heap_profile_path=/tmp/kudu.22552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.131
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:38.837699 22552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:38.838788 22552 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:38.848191 22560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:38.849541 22558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:38.850669 22557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:38.852068 22552 server_base.cc:1034] running on GCE node
I20240502 07:58:38.852545 22552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:38.853132 22552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:38.854293 22552 hybrid_clock.cc:648] HybridClock initialized: now 1714636718854260 us; error 58 us; skew 500 ppm
I20240502 07:58:38.856309 22552 webserver.cc:460] Webserver started at http://127.15.129.131:42309/ using document root <none> and password file <none>
I20240502 07:58:38.856871 22552 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:38.856926 22552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:38.857097 22552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:38.858772 22552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "e51b10af49484aaa8f0267aebc1bf27f"
format_stamp: "Formatted at 2024-05-02 07:58:38 on dist-test-slave-0hx2"
I20240502 07:58:38.859216 22552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "e51b10af49484aaa8f0267aebc1bf27f"
format_stamp: "Formatted at 2024-05-02 07:58:38 on dist-test-slave-0hx2"
I20240502 07:58:38.862411 22552 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20240502 07:58:38.864585 22566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:38.865612 22552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:38.865710 22552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "e51b10af49484aaa8f0267aebc1bf27f"
format_stamp: "Formatted at 2024-05-02 07:58:38 on dist-test-slave-0hx2"
I20240502 07:58:38.865798 22552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:38.878530 22552 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:38.879191 22552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:38.879350 22552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:38.879866 22552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:38.880756 22552 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:38.880815 22552 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:38.880883 22552 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:38.880911 22552 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:38.889669 22552 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.131:43703
I20240502 07:58:38.889725 22679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.131:43703 every 8 connection(s)
I20240502 07:58:38.890615 22552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 07:58:38.899431 22680 heartbeater.cc:347] Connected to a master server at 127.15.129.190:35417
I20240502 07:58:38.899684 22680 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:38.899942 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 22552
I20240502 07:58:38.900067 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 07:58:38.900297 22680 heartbeater.cc:511] Master 127.15.129.190:35417 requested a full tablet report, sending...
I20240502 07:58:38.901252 22230 ts_manager.cc:194] Registered new tserver with Master: e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:38.901942 22230 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.131:50851
I20240502 07:58:38.902274 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.132:0
--local_ip_for_outbound_sockets=127.15.129.132
--webserver_interface=127.15.129.132
--webserver_port=0
--tserver_master_addrs=127.15.129.190:35417
--builtin_ntp_servers=127.15.129.148:39703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 07:58:39.009723 22683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:39.009967 22683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:39.010041 22683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:39.013060 22683 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:39.013139 22683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:39.013245 22683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.132
I20240502 07:58:39.016866 22683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.15.129.132
--webserver_port=0
--tserver_master_addrs=127.15.129.190:35417
--never_fsync=true
--heap_profile_path=/tmp/kudu.22683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.132
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:39.017818 22683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:39.018903 22683 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:39.028587 22689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:39.029858 22688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:39.030849 22691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:39.032366 22683 server_base.cc:1034] running on GCE node
I20240502 07:58:39.032796 22683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:39.033459 22683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:39.034624 22683 hybrid_clock.cc:648] HybridClock initialized: now 1714636719034607 us; error 35 us; skew 500 ppm
I20240502 07:58:39.036589 22683 webserver.cc:460] Webserver started at http://127.15.129.132:40101/ using document root <none> and password file <none>
I20240502 07:58:39.037117 22683 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:39.037168 22683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:39.037351 22683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:39.038993 22683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "f00f3a1876444ad2bf9c919068ab8014"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:39.039438 22683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "f00f3a1876444ad2bf9c919068ab8014"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:39.042729 22683 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:58:39.045003 22697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:39.046192 22683 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:39.046299 22683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "f00f3a1876444ad2bf9c919068ab8014"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:39.046473 22683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:39.063095 22683 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:39.063724 22683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:39.063856 22683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:39.064335 22683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:39.065186 22683 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:39.065232 22683 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:39.065270 22683 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:39.065284 22683 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:39.074265 22683 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.132:41673
I20240502 07:58:39.074303 22810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.132:41673 every 8 connection(s)
I20240502 07:58:39.075161 22683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 07:58:39.078366 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 22683
I20240502 07:58:39.078511 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 07:58:39.080773 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.133:0
--local_ip_for_outbound_sockets=127.15.129.133
--webserver_interface=127.15.129.133
--webserver_port=0
--tserver_master_addrs=127.15.129.190:35417
--builtin_ntp_servers=127.15.129.148:39703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:39.085168 22811 heartbeater.cc:347] Connected to a master server at 127.15.129.190:35417
I20240502 07:58:39.085412 22811 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:39.085866 22811 heartbeater.cc:511] Master 127.15.129.190:35417 requested a full tablet report, sending...
I20240502 07:58:39.086814 22230 ts_manager.cc:194] Registered new tserver with Master: f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:39.087340 22230 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.132:40507
W20240502 07:58:39.188987 22814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:39.189201 22814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:39.189276 22814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:39.192240 22814 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:39.192312 22814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:39.192400 22814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.133
I20240502 07:58:39.195899 22814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.15.129.133
--webserver_port=0
--tserver_master_addrs=127.15.129.190:35417
--never_fsync=true
--heap_profile_path=/tmp/kudu.22814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.133
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:39.196760 22814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:39.197898 22814 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:39.206992 22820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:39.208281 22819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:39.209527 22822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:39.210834 22814 server_base.cc:1034] running on GCE node
I20240502 07:58:39.211269 22814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:39.211856 22814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:39.213027 22814 hybrid_clock.cc:648] HybridClock initialized: now 1714636719213014 us; error 26 us; skew 500 ppm
I20240502 07:58:39.214962 22814 webserver.cc:460] Webserver started at http://127.15.129.133:44545/ using document root <none> and password file <none>
I20240502 07:58:39.215513 22814 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:39.215586 22814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:39.215773 22814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:39.217514 22814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:39.218040 22814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:39.221555 22814 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:58:39.223757 22828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:39.224881 22814 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:39.224979 22814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:39.225082 22814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:39.246719 22814 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:39.247359 22814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:39.247506 22814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:39.248042 22814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:39.248916 22814 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:39.248966 22814 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:39.249028 22814 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:39.249058 22814 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:39.257472 22814 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.133:41661
I20240502 07:58:39.257526 22941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.133:41661 every 8 connection(s)
I20240502 07:58:39.258394 22814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 07:58:39.267616 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 22814
I20240502 07:58:39.267755 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 07:58:39.268210 22942 heartbeater.cc:347] Connected to a master server at 127.15.129.190:35417
I20240502 07:58:39.268430 22942 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:39.268927 22942 heartbeater.cc:511] Master 127.15.129.190:35417 requested a full tablet report, sending...
I20240502 07:58:39.269958 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.134:0
--local_ip_for_outbound_sockets=127.15.129.134
--webserver_interface=127.15.129.134
--webserver_port=0
--tserver_master_addrs=127.15.129.190:35417
--builtin_ntp_servers=127.15.129.148:39703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:39.269989 22230 ts_manager.cc:194] Registered new tserver with Master: 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:58:39.270696 22230 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.133:53957
W20240502 07:58:39.377662 22945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:39.377905 22945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:39.377979 22945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:39.381098 22945 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:39.381186 22945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:39.381278 22945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.134
I20240502 07:58:39.385257 22945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.134:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.15.129.134
--webserver_port=0
--tserver_master_addrs=127.15.129.190:35417
--never_fsync=true
--heap_profile_path=/tmp/kudu.22945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.134
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:39.386232 22945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:39.387374 22945 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:39.396623 22950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:39.397524 22953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:39.398874 22951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:39.399941 22945 server_base.cc:1034] running on GCE node
I20240502 07:58:39.400317 22945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:39.400875 22945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:39.402035 22945 hybrid_clock.cc:648] HybridClock initialized: now 1714636719402023 us; error 26 us; skew 500 ppm
I20240502 07:58:39.403916 22945 webserver.cc:460] Webserver started at http://127.15.129.134:46761/ using document root <none> and password file <none>
I20240502 07:58:39.404450 22945 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:39.404503 22945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:39.404667 22945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:39.406324 22945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "1aecef81733c47b3b468504934357231"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:39.406780 22945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "1aecef81733c47b3b468504934357231"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:39.409955 22945 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20240502 07:58:39.412135 22959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:39.413113 22945 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:39.413215 22945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "1aecef81733c47b3b468504934357231"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:39.413305 22945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:39.429046 22945 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:39.429744 22945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:39.429900 22945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:39.430424 22945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:39.431279 22945 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:39.431329 22945 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:39.431396 22945 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:39.431423 22945 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:39.439641 22945 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.134:35995
I20240502 07:58:39.439666 23072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.134:35995 every 8 connection(s)
I20240502 07:58:39.440543 22945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 07:58:39.445930 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 22945
I20240502 07:58:39.446065 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240502 07:58:39.448452 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.135:0
--local_ip_for_outbound_sockets=127.15.129.135
--webserver_interface=127.15.129.135
--webserver_port=0
--tserver_master_addrs=127.15.129.190:35417
--builtin_ntp_servers=127.15.129.148:39703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:39.449741 23073 heartbeater.cc:347] Connected to a master server at 127.15.129.190:35417
I20240502 07:58:39.449965 23073 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:39.450413 23073 heartbeater.cc:511] Master 127.15.129.190:35417 requested a full tablet report, sending...
I20240502 07:58:39.451391 22230 ts_manager.cc:194] Registered new tserver with Master: 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:58:39.451954 22230 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.134:38975
I20240502 07:58:39.520910 22418 heartbeater.cc:503] Master 127.15.129.190:35417 was elected leader, sending a full tablet report...
W20240502 07:58:39.556160 23076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:39.556407 23076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:39.556452 23076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:39.559505 23076 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:39.559577 23076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:39.559646 23076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.135
I20240502 07:58:39.563266 23076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.135:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.15.129.135
--webserver_port=0
--tserver_master_addrs=127.15.129.190:35417
--never_fsync=true
--heap_profile_path=/tmp/kudu.23076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.135
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:39.564224 23076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:39.565312 23076 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:39.574899 23082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:39.576224 23081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:39.577203 23084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:39.578382 23076 server_base.cc:1034] running on GCE node
I20240502 07:58:39.578822 23076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:39.579423 23076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:39.580569 23076 hybrid_clock.cc:648] HybridClock initialized: now 1714636719580546 us; error 40 us; skew 500 ppm
I20240502 07:58:39.582554 23076 webserver.cc:460] Webserver started at http://127.15.129.135:33487/ using document root <none> and password file <none>
I20240502 07:58:39.583122 23076 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:39.583195 23076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:39.583377 23076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:39.585068 23076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "227eec4b5f014d349661f6e0b39b9055"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:39.585559 23076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "227eec4b5f014d349661f6e0b39b9055"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:39.588886 23076 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20240502 07:58:39.591082 23090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:39.592063 23076 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:39.592167 23076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "227eec4b5f014d349661f6e0b39b9055"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:39.592250 23076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:39.627710 23076 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:39.628360 23076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:39.628518 23076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:39.629056 23076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:39.629976 23076 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:39.630024 23076 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:39.630086 23076 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:39.630115 23076 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:39.638479 23076 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.135:42733
I20240502 07:58:39.638509 23203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.135:42733 every 8 connection(s)
I20240502 07:58:39.639407 23076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 07:58:39.644912 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 23076
I20240502 07:58:39.645051 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240502 07:58:39.647267 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.136:0
--local_ip_for_outbound_sockets=127.15.129.136
--webserver_interface=127.15.129.136
--webserver_port=0
--tserver_master_addrs=127.15.129.190:35417
--builtin_ntp_servers=127.15.129.148:39703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:39.648289 23204 heartbeater.cc:347] Connected to a master server at 127.15.129.190:35417
I20240502 07:58:39.648545 23204 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:39.648998 23204 heartbeater.cc:511] Master 127.15.129.190:35417 requested a full tablet report, sending...
I20240502 07:58:39.650049 22230 ts_manager.cc:194] Registered new tserver with Master: 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:58:39.650828 22230 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.135:36565
I20240502 07:58:39.726583 22549 heartbeater.cc:503] Master 127.15.129.190:35417 was elected leader, sending a full tablet report...
W20240502 07:58:39.754658 23207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:39.754895 23207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:39.754968 23207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:39.758651 23207 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:39.758778 23207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:39.758888 23207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.136
I20240502 07:58:39.762785 23207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.136:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.15.129.136
--webserver_port=0
--tserver_master_addrs=127.15.129.190:35417
--never_fsync=true
--heap_profile_path=/tmp/kudu.23207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.136
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:39.763747 23207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:39.764873 23207 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:39.774467 23215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:39.775372 23212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:39.776791 23213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:39.778178 23207 server_base.cc:1034] running on GCE node
I20240502 07:58:39.778609 23207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:39.779186 23207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:39.780372 23207 hybrid_clock.cc:648] HybridClock initialized: now 1714636719780333 us; error 42 us; skew 500 ppm
I20240502 07:58:39.782614 23207 webserver.cc:460] Webserver started at http://127.15.129.136:46237/ using document root <none> and password file <none>
I20240502 07:58:39.783232 23207 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:39.783295 23207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:39.783505 23207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:58:39.785216 23207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "43930dee0cd5498fa8e357a34dd71432"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:39.785760 23207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "43930dee0cd5498fa8e357a34dd71432"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:39.789203 23207 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20240502 07:58:39.791653 23221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:39.792713 23207 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:39.792840 23207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "43930dee0cd5498fa8e357a34dd71432"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:39.792937 23207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:39.803140 23207 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:39.803763 23207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:39.803921 23207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:39.804454 23207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:39.805400 23207 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:39.805450 23207 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:39.805510 23207 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:39.805543 23207 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:39.813642 23207 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.136:36591
I20240502 07:58:39.813671 23334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.136:36591 every 8 connection(s)
I20240502 07:58:39.814570 23207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 07:58:39.823177 23335 heartbeater.cc:347] Connected to a master server at 127.15.129.190:35417
I20240502 07:58:39.823402 23335 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:39.823400 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 23207
I20240502 07:58:39.823503 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240502 07:58:39.823873 23335 heartbeater.cc:511] Master 127.15.129.190:35417 requested a full tablet report, sending...
I20240502 07:58:39.824870 22230 ts_manager.cc:194] Registered new tserver with Master: 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:58:39.825573 22230 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.136:46739
I20240502 07:58:39.825865 15878 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 07:58:39.835628 15878 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 07:58:39.835702 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 22814
I20240502 07:58:39.840914 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 22945
I20240502 07:58:39.846494 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 23076
I20240502 07:58:39.851953 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 23207
I20240502 07:58:39.904078 22680 heartbeater.cc:503] Master 127.15.129.190:35417 was elected leader, sending a full tablet report...
I20240502 07:58:40.089401 22811 heartbeater.cc:503] Master 127.15.129.190:35417 was elected leader, sending a full tablet report...
I20240502 07:58:43.608812 22230 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:43.629258 22482 tablet_service.cc:1467] Processing CreateTablet for tablet 41b11b805d4a40c5ab51e5e2ff5b6cf2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bc21302f92b248cf8c6d0a8d666ab5a4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:43.629648 22744 tablet_service.cc:1467] Processing CreateTablet for tablet 8ffeb8c1919746c3aaeaeb9ddefb8793 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bc21302f92b248cf8c6d0a8d666ab5a4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:43.630367 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ffeb8c1919746c3aaeaeb9ddefb8793. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:43.630360 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b11b805d4a40c5ab51e5e2ff5b6cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:43.631449 22613 tablet_service.cc:1467] Processing CreateTablet for tablet 8ffeb8c1919746c3aaeaeb9ddefb8793 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bc21302f92b248cf8c6d0a8d666ab5a4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:43.631449 22351 tablet_service.cc:1467] Processing CreateTablet for tablet 41b11b805d4a40c5ab51e5e2ff5b6cf2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bc21302f92b248cf8c6d0a8d666ab5a4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:43.632319 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ffeb8c1919746c3aaeaeb9ddefb8793. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:43.632319 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b11b805d4a40c5ab51e5e2ff5b6cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:43.633541 22614 tablet_service.cc:1467] Processing CreateTablet for tablet 8454cd827e5742819d06a9f883d161bb (DEFAULT_TABLE table=rebalance_test_table_0 [id=bc21302f92b248cf8c6d0a8d666ab5a4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:43.633680 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 8454cd827e5742819d06a9f883d161bb (DEFAULT_TABLE table=rebalance_test_table_0 [id=bc21302f92b248cf8c6d0a8d666ab5a4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:43.633790 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8454cd827e5742819d06a9f883d161bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:43.634004 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8454cd827e5742819d06a9f883d161bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:43.634743 22350 tablet_service.cc:1467] Processing CreateTablet for tablet 8ffeb8c1919746c3aaeaeb9ddefb8793 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bc21302f92b248cf8c6d0a8d666ab5a4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:43.635003 22350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ffeb8c1919746c3aaeaeb9ddefb8793. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:43.637532 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 8454cd827e5742819d06a9f883d161bb (DEFAULT_TABLE table=rebalance_test_table_0 [id=bc21302f92b248cf8c6d0a8d666ab5a4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:43.637842 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8454cd827e5742819d06a9f883d161bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:43.638456 23356 tablet_bootstrap.cc:492] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:43.639221 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 41b11b805d4a40c5ab51e5e2ff5b6cf2 (DEFAULT_TABLE table=rebalance_test_table_0 [id=bc21302f92b248cf8c6d0a8d666ab5a4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:43.639514 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b11b805d4a40c5ab51e5e2ff5b6cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:43.641026 23356 tablet_bootstrap.cc:654] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:43.641960 23356 log.cc:824] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:43.643088 23358 tablet_bootstrap.cc:492] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:43.644960 23359 tablet_bootstrap.cc:492] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:43.645206 23355 tablet_bootstrap.cc:492] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:43.645395 23358 tablet_bootstrap.cc:654] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:43.646008 23356 tablet_bootstrap.cc:492] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:43.646220 23356 ts_tablet_manager.cc:1367] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20240502 07:58:43.646401 23358 log.cc:824] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:43.647166 23355 tablet_bootstrap.cc:654] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:43.647171 23359 tablet_bootstrap.cc:654] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:43.647936 23355 log.cc:824] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:43.648177 23358 tablet_bootstrap.cc:492] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:43.648164 23359 log.cc:824] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:43.648358 23358 ts_tablet_manager.cc:1367] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.003s
I20240502 07:58:43.649704 23355 tablet_bootstrap.cc:492] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:43.649704 23359 tablet_bootstrap.cc:492] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:43.649863 23359 ts_tablet_manager.cc:1367] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240502 07:58:43.649881 23355 ts_tablet_manager.cc:1367] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20240502 07:58:43.649860 23356 raft_consensus.cc:348] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:43.650144 23356 raft_consensus.cc:374] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:43.650204 23356 raft_consensus.cc:729] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:43.652079 23356 consensus_queue.cc:260] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:43.652127 23358 raft_consensus.cc:348] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:43.652295 23358 raft_consensus.cc:374] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:43.652365 23358 raft_consensus.cc:729] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:43.652984 23356 ts_tablet_manager.cc:1398] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20240502 07:58:43.653430 23356 tablet_bootstrap.cc:492] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:43.653635 23355 raft_consensus.cc:348] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:43.653635 23359 raft_consensus.cc:348] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:43.653821 23359 raft_consensus.cc:374] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:43.653846 23355 raft_consensus.cc:374] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:43.653894 23359 raft_consensus.cc:729] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:43.653936 23355 raft_consensus.cc:729] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:43.654371 23359 consensus_queue.cc:260] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:43.654371 23355 consensus_queue.cc:260] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:43.654600 23356 tablet_bootstrap.cc:654] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:43.655222 23359 ts_tablet_manager.cc:1398] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 07:58:43.655222 23355 ts_tablet_manager.cc:1398] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 07:58:43.655581 23359 tablet_bootstrap.cc:492] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:43.655876 23356 tablet_bootstrap.cc:492] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:43.655951 23356 ts_tablet_manager.cc:1367] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:43.656522 23359 tablet_bootstrap.cc:654] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:43.656504 23356 raft_consensus.cc:348] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:43.656622 23356 raft_consensus.cc:374] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:43.656654 23356 raft_consensus.cc:729] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:43.656769 23356 consensus_queue.cc:260] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:43.657088 23356 ts_tablet_manager.cc:1398] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:43.657258 23356 tablet_bootstrap.cc:492] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:43.657269 23355 tablet_bootstrap.cc:492] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:43.657639 23358 consensus_queue.cc:260] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:43.658145 23356 tablet_bootstrap.cc:654] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:43.658296 23355 tablet_bootstrap.cc:654] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:43.658658 23358 ts_tablet_manager.cc:1398] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.010s user 0.003s sys 0.002s
I20240502 07:58:43.659804 23356 tablet_bootstrap.cc:492] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:43.659879 23356 ts_tablet_manager.cc:1367] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:43.660094 23359 tablet_bootstrap.cc:492] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:43.660164 23358 tablet_bootstrap.cc:492] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:43.660187 23359 ts_tablet_manager.cc:1367] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:43.660349 23356 raft_consensus.cc:348] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:43.660451 23356 raft_consensus.cc:374] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:43.660481 23356 raft_consensus.cc:729] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:43.660598 23356 consensus_queue.cc:260] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:43.660712 23359 raft_consensus.cc:348] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:43.660820 23359 raft_consensus.cc:374] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:43.660853 23359 raft_consensus.cc:729] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:43.660859 23356 ts_tablet_manager.cc:1398] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:43.660969 23359 consensus_queue.cc:260] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:43.661161 23358 tablet_bootstrap.cc:654] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:43.661401 23359 ts_tablet_manager.cc:1398] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:43.662392 23358 tablet_bootstrap.cc:492] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:43.662482 23358 ts_tablet_manager.cc:1367] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:43.662956 23358 raft_consensus.cc:348] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:43.663053 23358 raft_consensus.cc:374] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:43.663084 23358 raft_consensus.cc:729] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:43.663262 23358 consensus_queue.cc:260] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:43.663566 23358 ts_tablet_manager.cc:1398] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:43.664480 23355 tablet_bootstrap.cc:492] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:43.664577 23355 ts_tablet_manager.cc:1367] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:58:43.665131 23355 raft_consensus.cc:348] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:43.665222 23355 raft_consensus.cc:374] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:43.665244 23355 raft_consensus.cc:729] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:43.665371 23355 consensus_queue.cc:260] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:43.665623 23355 ts_tablet_manager.cc:1398] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:43.682492 23363 raft_consensus.cc:482] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:43.682678 23363 raft_consensus.cc:504] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:43.683779 23363 leader_election.cc:288] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:43.687623 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b11b805d4a40c5ab51e5e2ff5b6cf2" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:43.687620 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b11b805d4a40c5ab51e5e2ff5b6cf2" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:43.687896 22765 raft_consensus.cc:2454] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:43.687969 22503 raft_consensus.cc:2454] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:43.688426 22308 leader_election.cc:302] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:43.688529 23369 raft_consensus.cc:482] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:43.688710 23369 raft_consensus.cc:504] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:43.688861 23363 raft_consensus.cc:2781] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:43.688943 23363 raft_consensus.cc:482] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:43.688993 23363 raft_consensus.cc:3037] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:43.689726 23369 leader_election.cc:288] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:43.690016 23363 raft_consensus.cc:504] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:43.690416 23363 leader_election.cc:288] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:43.690928 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b11b805d4a40c5ab51e5e2ff5b6cf2" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:43.690950 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b11b805d4a40c5ab51e5e2ff5b6cf2" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:43.691053 22503 raft_consensus.cc:3037] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:43.691056 22765 raft_consensus.cc:3037] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:43.692003 22503 raft_consensus.cc:2454] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:43.692087 22765 raft_consensus.cc:2454] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:43.692406 22305 leader_election.cc:302] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:43.692687 23363 raft_consensus.cc:2781] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:43.692773 23363 raft_consensus.cc:686] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:58:43.693097 23363 consensus_queue.cc:237] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:43.694372 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8454cd827e5742819d06a9f883d161bb" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:43.694661 22634 raft_consensus.cc:2454] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:43.695087 22439 leader_election.cc:302] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:43.695340 23369 raft_consensus.cc:2781] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:43.695394 23369 raft_consensus.cc:482] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:43.695442 23369 raft_consensus.cc:3037] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:43.696169 23369 raft_consensus.cc:504] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:43.695941 22227 catalog_manager.cc:5484] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:58:43.696440 23369 leader_election.cc:288] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:43.697131 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8454cd827e5742819d06a9f883d161bb" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:43.697188 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8454cd827e5742819d06a9f883d161bb" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:43.697204 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8454cd827e5742819d06a9f883d161bb" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:43.697268 22634 raft_consensus.cc:3037] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:43.697350 22371 raft_consensus.cc:3037] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:43.698302 22634 raft_consensus.cc:2454] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:58:43.698302 22371 raft_consensus.cc:2454] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:58:43.698580 22439 leader_election.cc:302] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:43.698731 23369 raft_consensus.cc:2781] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:43.698940 23369 raft_consensus.cc:686] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:58:43.699237 23369 consensus_queue.cc:237] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:43.701850 22228 catalog_manager.cc:5484] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
W20240502 07:58:43.718133 22550 tablet.cc:2369] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 07:58:43.760223 22419 tablet.cc:2369] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:58:43.779204 23367 raft_consensus.cc:482] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:43.779341 23367 raft_consensus.cc:504] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:43.779789 23367 leader_election.cc:288] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:43.780681 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ffeb8c1919746c3aaeaeb9ddefb8793" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:43.780846 22765 raft_consensus.cc:2454] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:43.781132 22308 leader_election.cc:302] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:43.781344 23367 raft_consensus.cc:2781] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:43.781407 23367 raft_consensus.cc:482] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:43.781458 23367 raft_consensus.cc:3037] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:43.782495 23367 raft_consensus.cc:504] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:43.782876 23367 leader_election.cc:288] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:43.783216 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ffeb8c1919746c3aaeaeb9ddefb8793" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:43.783339 22765 raft_consensus.cc:3037] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:43.783923 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ffeb8c1919746c3aaeaeb9ddefb8793" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:43.784006 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ffeb8c1919746c3aaeaeb9ddefb8793" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:43.784060 22634 raft_consensus.cc:2454] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:43.784330 22765 raft_consensus.cc:2454] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:43.784572 22308 leader_election.cc:302] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters: e51b10af49484aaa8f0267aebc1bf27f
I20240502 07:58:43.784766 23367 raft_consensus.cc:2781] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:43.784864 23367 raft_consensus.cc:686] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:58:43.785014 23367 consensus_queue.cc:237] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:43.786689 22228 catalog_manager.cc:5484] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
W20240502 07:58:43.829492 22812 tablet.cc:2369] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 07:58:43.900844 22681 tablet.cc:2369] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 07:58:43.901335 23382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:43.901583 23382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:43.904727 23382 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240502 07:58:43.992830 22503 raft_consensus.cc:1261] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:43.993211 22765 raft_consensus.cc:1261] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:43.993762 22633 raft_consensus.cc:3037] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:43.993808 23367 consensus_queue.cc:1035] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:43.994779 22633 raft_consensus.cc:1261] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:43.995848 22371 raft_consensus.cc:1261] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:43.996291 23367 consensus_queue.cc:1035] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:43.996542 23369 consensus_queue.cc:1035] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:43.996730 23367 consensus_queue.cc:1035] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:44.003979 22633 raft_consensus.cc:1261] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:44.004437 23369 consensus_queue.cc:1035] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:44.008687 23411 mvcc.cc:205] Tried to move back new op lower bound from 7023152021471490048 to 7023152020625166336. Current Snapshot: MvccSnapshot[applied={T|T < 7023152021471490048}]
I20240502 07:58:44.012918 23416 mvcc.cc:205] Tried to move back new op lower bound from 7023152021471490048 to 7023152020625166336. Current Snapshot: MvccSnapshot[applied={T|T < 7023152021471490048}]
I20240502 07:58:44.013762 23413 mvcc.cc:205] Tried to move back new op lower bound from 7023152021468733440 to 7023152020249944064. Current Snapshot: MvccSnapshot[applied={T|T < 7023152021468733440}]
I20240502 07:58:44.014582 22764 raft_consensus.cc:1261] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:44.015187 23363 consensus_queue.cc:1035] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:44.034828 23414 mvcc.cc:205] Tried to move back new op lower bound from 7023152021471490048 to 7023152020625166336. Current Snapshot: MvccSnapshot[applied={T|T < 7023152021471490048}]
INSERT report
rows total: 2000
time total: 111.036 ms
time per row: 0.0555181 ms
I20240502 07:58:44.065079 22228 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:44.070482 22482 tablet_service.cc:1467] Processing CreateTablet for tablet d61cb2b6dce1409bb30d1c68c970dc22 (DEFAULT_TABLE table=rebalance_test_table_1 [id=be3f9b3a6e5147b0afe27972b50219ce]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:44.070586 22745 tablet_service.cc:1467] Processing CreateTablet for tablet d61cb2b6dce1409bb30d1c68c970dc22 (DEFAULT_TABLE table=rebalance_test_table_1 [id=be3f9b3a6e5147b0afe27972b50219ce]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:44.070796 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d61cb2b6dce1409bb30d1c68c970dc22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.070868 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d61cb2b6dce1409bb30d1c68c970dc22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.072434 23355 tablet_bootstrap.cc:492] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:44.072463 23358 tablet_bootstrap.cc:492] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:44.072912 22613 tablet_service.cc:1467] Processing CreateTablet for tablet d61cb2b6dce1409bb30d1c68c970dc22 (DEFAULT_TABLE table=rebalance_test_table_1 [id=be3f9b3a6e5147b0afe27972b50219ce]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:44.073194 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d61cb2b6dce1409bb30d1c68c970dc22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.073253 23358 tablet_bootstrap.cc:654] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:44.073267 23355 tablet_bootstrap.cc:654] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:44.074170 22745 tablet_service.cc:1467] Processing CreateTablet for tablet c5841aadb46447e685af47b70d0feff4 (DEFAULT_TABLE table=rebalance_test_table_1 [id=be3f9b3a6e5147b0afe27972b50219ce]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:44.074282 22614 tablet_service.cc:1467] Processing CreateTablet for tablet c5841aadb46447e685af47b70d0feff4 (DEFAULT_TABLE table=rebalance_test_table_1 [id=be3f9b3a6e5147b0afe27972b50219ce]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:44.074395 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5841aadb46447e685af47b70d0feff4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.074455 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5841aadb46447e685af47b70d0feff4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.075428 23359 tablet_bootstrap.cc:492] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:44.076237 23359 tablet_bootstrap.cc:654] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:44.076511 22350 tablet_service.cc:1467] Processing CreateTablet for tablet ce0f9b48d2db4855a86eb20955fdc853 (DEFAULT_TABLE table=rebalance_test_table_1 [id=be3f9b3a6e5147b0afe27972b50219ce]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:44.076753 22350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce0f9b48d2db4855a86eb20955fdc853. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.077116 23355 tablet_bootstrap.cc:492] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:44.077178 23355 ts_tablet_manager.cc:1367] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:58:44.077615 23355 raft_consensus.cc:348] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:44.077734 23355 raft_consensus.cc:374] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:44.077760 23355 raft_consensus.cc:729] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:44.077869 23355 consensus_queue.cc:260] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:44.078114 23355 ts_tablet_manager.cc:1398] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:44.078253 23355 tablet_bootstrap.cc:492] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:44.078685 23356 tablet_bootstrap.cc:492] T ce0f9b48d2db4855a86eb20955fdc853 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:44.079105 23355 tablet_bootstrap.cc:654] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:44.079454 23356 tablet_bootstrap.cc:654] T ce0f9b48d2db4855a86eb20955fdc853 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:44.080740 23355 tablet_bootstrap.cc:492] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:44.080817 23355 ts_tablet_manager.cc:1367] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:44.080857 23359 tablet_bootstrap.cc:492] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:44.080926 23359 ts_tablet_manager.cc:1367] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240502 07:58:44.081241 23355 raft_consensus.cc:348] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:44.081360 23355 raft_consensus.cc:374] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:44.081394 23355 raft_consensus.cc:729] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:44.081355 23359 raft_consensus.cc:348] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:44.081470 23359 raft_consensus.cc:374] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:44.081501 23359 raft_consensus.cc:729] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:44.081504 23355 consensus_queue.cc:260] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:44.081609 23359 consensus_queue.cc:260] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:44.081693 22482 tablet_service.cc:1467] Processing CreateTablet for tablet c5841aadb46447e685af47b70d0feff4 (DEFAULT_TABLE table=rebalance_test_table_1 [id=be3f9b3a6e5147b0afe27972b50219ce]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:44.081764 23355 ts_tablet_manager.cc:1398] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:44.081894 23359 ts_tablet_manager.cc:1398] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:44.081933 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5841aadb46447e685af47b70d0feff4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.082057 23359 tablet_bootstrap.cc:492] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:44.082976 23359 tablet_bootstrap.cc:654] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:44.083756 22483 tablet_service.cc:1467] Processing CreateTablet for tablet ce0f9b48d2db4855a86eb20955fdc853 (DEFAULT_TABLE table=rebalance_test_table_1 [id=be3f9b3a6e5147b0afe27972b50219ce]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:44.083992 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce0f9b48d2db4855a86eb20955fdc853. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.084049 23359 tablet_bootstrap.cc:492] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:44.084111 23359 ts_tablet_manager.cc:1367] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:44.084486 23359 raft_consensus.cc:348] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:44.084569 23359 raft_consensus.cc:374] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:44.084525 22614 tablet_service.cc:1467] Processing CreateTablet for tablet ce0f9b48d2db4855a86eb20955fdc853 (DEFAULT_TABLE table=rebalance_test_table_1 [id=be3f9b3a6e5147b0afe27972b50219ce]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:44.084600 23359 raft_consensus.cc:729] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:44.084725 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce0f9b48d2db4855a86eb20955fdc853. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.084738 23359 consensus_queue.cc:260] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:44.084988 23359 ts_tablet_manager.cc:1398] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:44.086999 23359 tablet_bootstrap.cc:492] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:44.087771 23359 tablet_bootstrap.cc:654] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:44.088035 23356 tablet_bootstrap.cc:492] T ce0f9b48d2db4855a86eb20955fdc853 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:44.088135 23356 ts_tablet_manager.cc:1367] T ce0f9b48d2db4855a86eb20955fdc853 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 07:58:44.088546 23356 raft_consensus.cc:348] T ce0f9b48d2db4855a86eb20955fdc853 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:44.088675 23356 raft_consensus.cc:374] T ce0f9b48d2db4855a86eb20955fdc853 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:44.088721 23356 raft_consensus.cc:729] T ce0f9b48d2db4855a86eb20955fdc853 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:44.088856 23356 consensus_queue.cc:260] T ce0f9b48d2db4855a86eb20955fdc853 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:44.088997 23359 tablet_bootstrap.cc:492] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:44.089078 23359 ts_tablet_manager.cc:1367] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:44.089149 23356 ts_tablet_manager.cc:1398] T ce0f9b48d2db4855a86eb20955fdc853 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:44.089192 23358 tablet_bootstrap.cc:492] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:44.089262 23358 ts_tablet_manager.cc:1367] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 07:58:44.089666 23358 raft_consensus.cc:348] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:44.089759 23358 raft_consensus.cc:374] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:44.089787 23358 raft_consensus.cc:729] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:44.089884 23358 consensus_queue.cc:260] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:44.090103 23358 ts_tablet_manager.cc:1398] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:44.090238 23358 tablet_bootstrap.cc:492] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:44.089964 23359 raft_consensus.cc:348] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:44.090900 23359 raft_consensus.cc:374] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:44.090950 23359 raft_consensus.cc:729] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:44.090974 23358 tablet_bootstrap.cc:654] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:44.091089 23359 consensus_queue.cc:260] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:44.091382 23359 ts_tablet_manager.cc:1398] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:44.093106 23358 tablet_bootstrap.cc:492] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:44.093195 23358 ts_tablet_manager.cc:1367] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:44.093632 23358 raft_consensus.cc:348] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:44.093711 23358 raft_consensus.cc:374] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:44.093735 23358 raft_consensus.cc:729] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:44.093847 23358 consensus_queue.cc:260] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:44.094143 23358 ts_tablet_manager.cc:1398] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:44.094723 23358 tablet_bootstrap.cc:492] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:44.095539 23358 tablet_bootstrap.cc:654] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:44.096614 23358 tablet_bootstrap.cc:492] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:44.096693 23358 ts_tablet_manager.cc:1367] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:58:44.097121 23358 raft_consensus.cc:348] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:44.097241 23358 raft_consensus.cc:374] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:44.097273 23358 raft_consensus.cc:729] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:44.097441 23358 consensus_queue.cc:260] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:44.097684 23358 ts_tablet_manager.cc:1398] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:44.128536 23372 raft_consensus.cc:482] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:44.129127 23372 raft_consensus.cc:504] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:44.130440 23372 leader_election.cc:288] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:44.134086 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61cb2b6dce1409bb30d1c68c970dc22" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:44.134174 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61cb2b6dce1409bb30d1c68c970dc22" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:44.134248 22503 raft_consensus.cc:2454] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:44.134315 22765 raft_consensus.cc:2454] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:44.134611 22567 leader_election.cc:302] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:44.134922 23372 raft_consensus.cc:2781] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:44.134979 23372 raft_consensus.cc:482] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:44.135006 23372 raft_consensus.cc:3037] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:44.135833 23372 raft_consensus.cc:504] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:44.136178 23372 leader_election.cc:288] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:44.136518 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61cb2b6dce1409bb30d1c68c970dc22" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:44.136538 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61cb2b6dce1409bb30d1c68c970dc22" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:44.136647 22503 raft_consensus.cc:3037] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:44.136656 22765 raft_consensus.cc:3037] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:44.137667 22503 raft_consensus.cc:2454] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:44.137667 22765 raft_consensus.cc:2454] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:44.138012 22570 leader_election.cc:302] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:44.138185 23372 raft_consensus.cc:2781] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:44.138294 23372 raft_consensus.cc:686] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:44.138604 23372 consensus_queue.cc:237] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:44.140686 22227 catalog_manager.cc:5484] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:58:44.224948 23372 raft_consensus.cc:482] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:44.225101 23372 raft_consensus.cc:504] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:44.225651 23372 leader_election.cc:288] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:44.226161 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce0f9b48d2db4855a86eb20955fdc853" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:44.226318 22503 raft_consensus.cc:2454] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:44.226621 22567 leader_election.cc:302] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:44.226789 23372 raft_consensus.cc:2781] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:44.226840 23372 raft_consensus.cc:482] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:44.226882 23372 raft_consensus.cc:3037] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:44.227773 23372 raft_consensus.cc:504] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:44.228251 23372 leader_election.cc:288] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:44.228473 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce0f9b48d2db4855a86eb20955fdc853" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:44.228593 22503 raft_consensus.cc:3037] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:44.228962 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce0f9b48d2db4855a86eb20955fdc853" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:44.229008 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce0f9b48d2db4855a86eb20955fdc853" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:44.229099 22371 raft_consensus.cc:2454] T ce0f9b48d2db4855a86eb20955fdc853 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:44.229573 22503 raft_consensus.cc:2454] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:44.229820 22567 leader_election.cc:302] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters: b533a7d8d97943609c6e28e3e36b78a4
I20240502 07:58:44.229974 23372 raft_consensus.cc:2781] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:44.230065 23372 raft_consensus.cc:686] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:44.230243 23372 consensus_queue.cc:237] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:44.231801 22227 catalog_manager.cc:5484] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:58:44.304888 23372 raft_consensus.cc:482] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:44.305037 23372 raft_consensus.cc:504] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:44.305502 23372 leader_election.cc:288] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:44.306104 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5841aadb46447e685af47b70d0feff4" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:44.306104 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5841aadb46447e685af47b70d0feff4" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:44.306267 22765 raft_consensus.cc:2454] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:44.306263 22503 raft_consensus.cc:2454] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:44.306581 22567 leader_election.cc:302] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:44.306749 23372 raft_consensus.cc:2781] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:44.306809 23372 raft_consensus.cc:482] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:44.306854 23372 raft_consensus.cc:3037] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:44.307649 23372 raft_consensus.cc:504] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:44.307963 23372 leader_election.cc:288] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:44.308418 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5841aadb46447e685af47b70d0feff4" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:44.308418 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5841aadb46447e685af47b70d0feff4" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:44.308540 22765 raft_consensus.cc:3037] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:44.308540 22503 raft_consensus.cc:3037] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:44.309489 22503 raft_consensus.cc:2454] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:44.309489 22765 raft_consensus.cc:2454] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:44.309825 22570 leader_election.cc:302] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:44.309994 23372 raft_consensus.cc:2781] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:44.310099 23372 raft_consensus.cc:686] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:44.310230 23372 consensus_queue.cc:237] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:44.311863 22227 catalog_manager.cc:5484] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
W20240502 07:58:44.420753 23455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:44.421017 23455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:44.424109 23455 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240502 07:58:44.502375 22503 raft_consensus.cc:1261] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:44.503262 23372 consensus_queue.cc:1035] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:44.505498 22501 raft_consensus.cc:1261] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:44.507158 22503 raft_consensus.cc:1261] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:44.508008 22765 raft_consensus.cc:1261] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:44.508456 23482 consensus_queue.cc:1035] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:44.508576 22764 raft_consensus.cc:1261] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:44.508980 23365 consensus_queue.cc:1035] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:44.509064 22371 raft_consensus.cc:3037] T ce0f9b48d2db4855a86eb20955fdc853 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:44.510071 22371 raft_consensus.cc:1261] T ce0f9b48d2db4855a86eb20955fdc853 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:44.512048 23365 consensus_queue.cc:1035] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:44.516116 23365 consensus_queue.cc:1035] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:44.516160 23372 consensus_queue.cc:1035] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 98.0261 ms
time per row: 0.049013 ms
I20240502 07:58:44.567449 22230 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:44.573516 22745 tablet_service.cc:1467] Processing CreateTablet for tablet b005f3f379ba43209b105632ba773165 (DEFAULT_TABLE table=rebalance_test_table_2 [id=0937fa4568b0489ebd23bdd0b698a759]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:44.573848 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b005f3f379ba43209b105632ba773165. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.573988 22614 tablet_service.cc:1467] Processing CreateTablet for tablet dc9be64f2ead4130a9f1f0694131640f (DEFAULT_TABLE table=rebalance_test_table_2 [id=0937fa4568b0489ebd23bdd0b698a759]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:44.574278 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc9be64f2ead4130a9f1f0694131640f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.574560 22351 tablet_service.cc:1467] Processing CreateTablet for tablet 0c6512754f394ad2a6bca0bd276a76b8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=0937fa4568b0489ebd23bdd0b698a759]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:44.574729 22350 tablet_service.cc:1467] Processing CreateTablet for tablet b005f3f379ba43209b105632ba773165 (DEFAULT_TABLE table=rebalance_test_table_2 [id=0937fa4568b0489ebd23bdd0b698a759]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:44.574820 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c6512754f394ad2a6bca0bd276a76b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.574939 22350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b005f3f379ba43209b105632ba773165. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.575963 23355 tablet_bootstrap.cc:492] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:44.576344 23359 tablet_bootstrap.cc:492] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:44.574563 22352 tablet_service.cc:1467] Processing CreateTablet for tablet dc9be64f2ead4130a9f1f0694131640f (DEFAULT_TABLE table=rebalance_test_table_2 [id=0937fa4568b0489ebd23bdd0b698a759]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:44.576889 23355 tablet_bootstrap.cc:654] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:44.576996 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc9be64f2ead4130a9f1f0694131640f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.577176 23359 tablet_bootstrap.cc:654] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:44.577780 23356 tablet_bootstrap.cc:492] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:44.578476 23356 tablet_bootstrap.cc:654] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:44.579309 22483 tablet_service.cc:1467] Processing CreateTablet for tablet b005f3f379ba43209b105632ba773165 (DEFAULT_TABLE table=rebalance_test_table_2 [id=0937fa4568b0489ebd23bdd0b698a759]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:44.579342 22482 tablet_service.cc:1467] Processing CreateTablet for tablet 0c6512754f394ad2a6bca0bd276a76b8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=0937fa4568b0489ebd23bdd0b698a759]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:44.579439 23359 tablet_bootstrap.cc:492] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:44.579443 23355 tablet_bootstrap.cc:492] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:44.579515 23359 ts_tablet_manager.cc:1367] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 07:58:44.579520 23355 ts_tablet_manager.cc:1367] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:44.579582 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b005f3f379ba43209b105632ba773165. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.579793 23356 tablet_bootstrap.cc:492] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:44.579859 23356 ts_tablet_manager.cc:1367] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:44.579952 23355 raft_consensus.cc:348] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:44.579952 23359 raft_consensus.cc:348] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:44.580047 23355 raft_consensus.cc:374] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:44.580049 23359 raft_consensus.cc:374] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:44.580075 23355 raft_consensus.cc:729] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:44.580080 23359 raft_consensus.cc:729] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:44.580186 23355 consensus_queue.cc:260] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:44.580195 23359 consensus_queue.cc:260] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:44.580235 23356 raft_consensus.cc:348] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:44.580313 23356 raft_consensus.cc:374] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:44.580343 23356 raft_consensus.cc:729] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:44.580451 23356 consensus_queue.cc:260] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:44.580538 23359 ts_tablet_manager.cc:1398] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:44.580689 23356 ts_tablet_manager.cc:1398] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:44.580845 23356 tablet_bootstrap.cc:492] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:44.581651 23356 tablet_bootstrap.cc:654] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:44.581737 22744 tablet_service.cc:1467] Processing CreateTablet for tablet 0c6512754f394ad2a6bca0bd276a76b8 (DEFAULT_TABLE table=rebalance_test_table_2 [id=0937fa4568b0489ebd23bdd0b698a759]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:44.581951 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c6512754f394ad2a6bca0bd276a76b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.582904 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c6512754f394ad2a6bca0bd276a76b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.582959 23358 tablet_bootstrap.cc:492] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:44.583823 23358 tablet_bootstrap.cc:654] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:44.584295 23355 ts_tablet_manager.cc:1398] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:58:44.584461 23355 tablet_bootstrap.cc:492] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:44.585367 23355 tablet_bootstrap.cc:654] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:44.586413 23355 tablet_bootstrap.cc:492] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:44.586495 23355 ts_tablet_manager.cc:1367] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:58:44.587101 23355 raft_consensus.cc:348] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:44.587196 23355 raft_consensus.cc:374] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:44.587231 23355 raft_consensus.cc:729] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:44.587463 22745 tablet_service.cc:1467] Processing CreateTablet for tablet dc9be64f2ead4130a9f1f0694131640f (DEFAULT_TABLE table=rebalance_test_table_2 [id=0937fa4568b0489ebd23bdd0b698a759]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:44.587638 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc9be64f2ead4130a9f1f0694131640f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:44.587962 23355 consensus_queue.cc:260] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:44.588266 23355 ts_tablet_manager.cc:1398] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:44.588652 23358 tablet_bootstrap.cc:492] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:44.588730 23358 ts_tablet_manager.cc:1367] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240502 07:58:44.589107 23358 raft_consensus.cc:348] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:44.589191 23358 raft_consensus.cc:374] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:44.589213 23358 raft_consensus.cc:729] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:44.589313 23358 consensus_queue.cc:260] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:44.589529 23356 tablet_bootstrap.cc:492] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:44.589607 23358 ts_tablet_manager.cc:1398] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:44.589717 23356 ts_tablet_manager.cc:1367] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 07:58:44.589772 23358 tablet_bootstrap.cc:492] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:44.590065 23356 raft_consensus.cc:348] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:44.590147 23356 raft_consensus.cc:374] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:44.590167 23356 raft_consensus.cc:729] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:44.590262 23356 consensus_queue.cc:260] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:44.590499 23358 tablet_bootstrap.cc:654] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:44.590526 23356 ts_tablet_manager.cc:1398] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:44.590965 23355 tablet_bootstrap.cc:492] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:44.591590 23358 tablet_bootstrap.cc:492] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:44.591660 23358 ts_tablet_manager.cc:1367] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:58:44.591872 23355 tablet_bootstrap.cc:654] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:44.592106 23358 raft_consensus.cc:348] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:44.592262 23358 raft_consensus.cc:374] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:44.592319 23358 raft_consensus.cc:729] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:44.592510 23358 consensus_queue.cc:260] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:44.592813 23358 ts_tablet_manager.cc:1398] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:44.592882 23355 tablet_bootstrap.cc:492] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:44.592954 23355 ts_tablet_manager.cc:1367] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:44.593360 23355 raft_consensus.cc:348] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:44.593461 23355 raft_consensus.cc:374] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:44.593492 23355 raft_consensus.cc:729] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:44.593606 23355 consensus_queue.cc:260] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:44.593870 23355 ts_tablet_manager.cc:1398] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:44.593818 23356 tablet_bootstrap.cc:492] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:44.595237 23356 tablet_bootstrap.cc:654] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:44.596628 23356 tablet_bootstrap.cc:492] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:44.596696 23356 ts_tablet_manager.cc:1367] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:44.597100 23356 raft_consensus.cc:348] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:44.597210 23356 raft_consensus.cc:374] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:44.597252 23356 raft_consensus.cc:729] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:44.597441 23356 consensus_queue.cc:260] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:44.597733 23356 ts_tablet_manager.cc:1398] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:44.599933 23431 raft_consensus.cc:482] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:44.600021 23431 raft_consensus.cc:504] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:44.601094 23431 leader_election.cc:288] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:44.605307 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9be64f2ead4130a9f1f0694131640f" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:44.605494 22371 raft_consensus.cc:2454] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:44.605832 22701 leader_election.cc:302] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:44.605890 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9be64f2ead4130a9f1f0694131640f" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:44.606022 22633 raft_consensus.cc:2454] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:44.606078 23431 raft_consensus.cc:2781] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:44.606135 23431 raft_consensus.cc:482] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:44.606182 23431 raft_consensus.cc:3037] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:44.606976 23431 raft_consensus.cc:504] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:44.607265 23431 leader_election.cc:288] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:44.607600 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9be64f2ead4130a9f1f0694131640f" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:44.607702 22633 raft_consensus.cc:3037] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:44.607872 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc9be64f2ead4130a9f1f0694131640f" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:44.607981 22371 raft_consensus.cc:3037] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:44.608641 22633 raft_consensus.cc:2454] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:44.608915 22701 leader_election.cc:302] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:44.608924 22371 raft_consensus.cc:2454] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:44.609112 23431 raft_consensus.cc:2781] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:44.609365 23431 raft_consensus.cc:686] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:58:44.609671 23431 consensus_queue.cc:237] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:44.611661 22222 catalog_manager.cc:5484] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:58:44.674201 23369 raft_consensus.cc:482] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:44.674335 23369 raft_consensus.cc:504] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:44.674825 23369 leader_election.cc:288] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:44.675256 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c6512754f394ad2a6bca0bd276a76b8" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:44.675426 22371 raft_consensus.cc:2454] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:44.675733 22439 leader_election.cc:302] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:44.675899 23369 raft_consensus.cc:2781] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:44.675951 23369 raft_consensus.cc:482] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:44.675998 23369 raft_consensus.cc:3037] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:44.677040 23369 raft_consensus.cc:504] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:44.677417 23369 leader_election.cc:288] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:44.677767 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c6512754f394ad2a6bca0bd276a76b8" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:44.677892 22371 raft_consensus.cc:3037] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:44.678918 22371 raft_consensus.cc:2454] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:58:44.679214 22439 leader_election.cc:302] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:44.679354 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c6512754f394ad2a6bca0bd276a76b8" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:44.679399 23369 raft_consensus.cc:2781] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:44.679399 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c6512754f394ad2a6bca0bd276a76b8" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:44.679490 23369 raft_consensus.cc:686] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:58:44.679500 22764 raft_consensus.cc:2454] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:44.679648 23369 consensus_queue.cc:237] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:44.681442 22222 catalog_manager.cc:5484] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:58:44.853300 23431 raft_consensus.cc:482] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:44.853467 23431 raft_consensus.cc:504] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:44.853916 23431 leader_election.cc:288] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:44.854430 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b005f3f379ba43209b105632ba773165" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:44.854593 22371 raft_consensus.cc:2454] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:44.854921 22701 leader_election.cc:302] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:44.855111 23431 raft_consensus.cc:2781] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:44.855171 23431 raft_consensus.cc:482] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:44.855226 23431 raft_consensus.cc:3037] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:44.856266 23431 raft_consensus.cc:504] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:44.856626 23431 leader_election.cc:288] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:44.857028 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b005f3f379ba43209b105632ba773165" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:44.857158 22371 raft_consensus.cc:3037] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:44.857825 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b005f3f379ba43209b105632ba773165" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:44.857847 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b005f3f379ba43209b105632ba773165" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:44.857966 22371 raft_consensus.cc:2454] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:44.857982 22501 raft_consensus.cc:2454] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:44.858240 22701 leader_election.cc:302] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:44.858404 23431 raft_consensus.cc:2781] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:44.858475 23431 raft_consensus.cc:686] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:58:44.858613 23431 consensus_queue.cc:237] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:44.860291 22222 catalog_manager.cc:5484] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
W20240502 07:58:44.975492 23515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:44.975754 23515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:44.979040 23515 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240502 07:58:45.057348 22764 raft_consensus.cc:3037] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.058274 22633 raft_consensus.cc:1261] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:45.058455 22764 raft_consensus.cc:1261] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:45.058715 23508 consensus_queue.cc:1035] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:45.058802 22371 raft_consensus.cc:1261] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:45.060235 22502 raft_consensus.cc:3037] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.060537 23369 consensus_queue.cc:1035] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:45.060842 23369 consensus_queue.cc:1035] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:45.061055 22371 raft_consensus.cc:1261] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:45.061302 22502 raft_consensus.cc:1261] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:45.061545 23431 consensus_queue.cc:1035] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:45.063189 23508 consensus_queue.cc:1035] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:45.064914 22370 raft_consensus.cc:1261] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:45.065405 23508 consensus_queue.cc:1035] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 109.596 ms
time per row: 0.0547979 ms
I20240502 07:58:45.131350 22222 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:45.137112 22482 tablet_service.cc:1467] Processing CreateTablet for tablet f5f0690b92c140c49fe10ff46e0136df (DEFAULT_TABLE table=rebalance_test_table_3 [id=2c887f37ddf349f3bbec2b1330f05fc7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:45.137498 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5f0690b92c140c49fe10ff46e0136df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:45.137511 22352 tablet_service.cc:1467] Processing CreateTablet for tablet f5f0690b92c140c49fe10ff46e0136df (DEFAULT_TABLE table=rebalance_test_table_3 [id=2c887f37ddf349f3bbec2b1330f05fc7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:45.137809 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5f0690b92c140c49fe10ff46e0136df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:45.138047 22614 tablet_service.cc:1467] Processing CreateTablet for tablet f5f0690b92c140c49fe10ff46e0136df (DEFAULT_TABLE table=rebalance_test_table_3 [id=2c887f37ddf349f3bbec2b1330f05fc7]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:45.138298 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5f0690b92c140c49fe10ff46e0136df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:45.138572 22612 tablet_service.cc:1467] Processing CreateTablet for tablet f1890c9d5c664106bacc97d440c9cbc2 (DEFAULT_TABLE table=rebalance_test_table_3 [id=2c887f37ddf349f3bbec2b1330f05fc7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:45.138805 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1890c9d5c664106bacc97d440c9cbc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:45.139906 22351 tablet_service.cc:1467] Processing CreateTablet for tablet d9c0c75053be41f59786f1cf4fb3062c (DEFAULT_TABLE table=rebalance_test_table_3 [id=2c887f37ddf349f3bbec2b1330f05fc7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:45.140138 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9c0c75053be41f59786f1cf4fb3062c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:45.140211 23574 tablet_bootstrap.cc:492] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:45.140877 22613 tablet_service.cc:1467] Processing CreateTablet for tablet d9c0c75053be41f59786f1cf4fb3062c (DEFAULT_TABLE table=rebalance_test_table_3 [id=2c887f37ddf349f3bbec2b1330f05fc7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:45.141089 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9c0c75053be41f59786f1cf4fb3062c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:45.141139 23574 tablet_bootstrap.cc:654] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:45.141245 23575 tablet_bootstrap.cc:492] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:45.142138 22745 tablet_service.cc:1467] Processing CreateTablet for tablet d9c0c75053be41f59786f1cf4fb3062c (DEFAULT_TABLE table=rebalance_test_table_3 [id=2c887f37ddf349f3bbec2b1330f05fc7]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:45.142313 23575 tablet_bootstrap.cc:654] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:45.142391 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9c0c75053be41f59786f1cf4fb3062c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:45.142663 23574 tablet_bootstrap.cc:492] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:45.142745 23574 ts_tablet_manager.cc:1367] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:45.143052 22744 tablet_service.cc:1467] Processing CreateTablet for tablet f1890c9d5c664106bacc97d440c9cbc2 (DEFAULT_TABLE table=rebalance_test_table_3 [id=2c887f37ddf349f3bbec2b1330f05fc7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:45.143177 23574 raft_consensus.cc:348] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.143275 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1890c9d5c664106bacc97d440c9cbc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:45.143294 23574 raft_consensus.cc:374] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:45.143333 23574 raft_consensus.cc:729] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:45.143460 23574 consensus_queue.cc:260] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.143719 23574 ts_tablet_manager.cc:1398] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:45.143831 23574 tablet_bootstrap.cc:492] T d9c0c75053be41f59786f1cf4fb3062c P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:45.144447 23578 tablet_bootstrap.cc:492] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:45.144500 23574 tablet_bootstrap.cc:654] T d9c0c75053be41f59786f1cf4fb3062c P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:45.145684 23578 tablet_bootstrap.cc:654] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:45.145875 23573 tablet_bootstrap.cc:492] T f5f0690b92c140c49fe10ff46e0136df P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:45.145984 22482 tablet_service.cc:1467] Processing CreateTablet for tablet f1890c9d5c664106bacc97d440c9cbc2 (DEFAULT_TABLE table=rebalance_test_table_3 [id=2c887f37ddf349f3bbec2b1330f05fc7]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:45.146200 23575 tablet_bootstrap.cc:492] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:45.146226 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1890c9d5c664106bacc97d440c9cbc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:45.146286 23575 ts_tablet_manager.cc:1367] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:58:45.146690 23575 raft_consensus.cc:348] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.146800 23573 tablet_bootstrap.cc:654] T f5f0690b92c140c49fe10ff46e0136df P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:45.146802 23575 raft_consensus.cc:374] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:45.146835 23575 raft_consensus.cc:729] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:45.146970 23575 consensus_queue.cc:260] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.147296 23575 ts_tablet_manager.cc:1398] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:45.147513 23575 tablet_bootstrap.cc:492] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:45.147902 23574 tablet_bootstrap.cc:492] T d9c0c75053be41f59786f1cf4fb3062c P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:45.147979 23574 ts_tablet_manager.cc:1367] T d9c0c75053be41f59786f1cf4fb3062c P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:45.148391 23574 raft_consensus.cc:348] T d9c0c75053be41f59786f1cf4fb3062c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:45.148540 23574 raft_consensus.cc:374] T d9c0c75053be41f59786f1cf4fb3062c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:45.148579 23575 tablet_bootstrap.cc:654] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:45.148602 23574 raft_consensus.cc:729] T d9c0c75053be41f59786f1cf4fb3062c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:45.148770 23574 consensus_queue.cc:260] T d9c0c75053be41f59786f1cf4fb3062c P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:45.149058 23573 tablet_bootstrap.cc:492] T f5f0690b92c140c49fe10ff46e0136df P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:45.149128 23574 ts_tablet_manager.cc:1398] T d9c0c75053be41f59786f1cf4fb3062c P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:45.149281 23573 ts_tablet_manager.cc:1367] T f5f0690b92c140c49fe10ff46e0136df P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:45.149557 23575 tablet_bootstrap.cc:492] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:45.149636 23575 ts_tablet_manager.cc:1367] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:45.150060 23575 raft_consensus.cc:348] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:45.150154 23575 raft_consensus.cc:374] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:45.150189 23575 raft_consensus.cc:729] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:45.150313 23578 tablet_bootstrap.cc:492] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:45.150307 23575 consensus_queue.cc:260] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:45.150324 23573 raft_consensus.cc:348] T f5f0690b92c140c49fe10ff46e0136df P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.150429 23578 ts_tablet_manager.cc:1367] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:58:45.150437 23573 raft_consensus.cc:374] T f5f0690b92c140c49fe10ff46e0136df P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:45.150470 23573 raft_consensus.cc:729] T f5f0690b92c140c49fe10ff46e0136df P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:45.150588 23575 ts_tablet_manager.cc:1398] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:45.150727 23575 tablet_bootstrap.cc:492] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:45.150875 23578 raft_consensus.cc:348] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:45.150967 23578 raft_consensus.cc:374] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:45.151010 23578 raft_consensus.cc:729] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:45.151158 23578 consensus_queue.cc:260] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:45.151505 23575 tablet_bootstrap.cc:654] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:45.151829 23578 ts_tablet_manager.cc:1398] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:45.151984 23578 tablet_bootstrap.cc:492] T f1890c9d5c664106bacc97d440c9cbc2 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:45.152734 23575 tablet_bootstrap.cc:492] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:45.152807 23575 ts_tablet_manager.cc:1367] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:45.153075 23578 tablet_bootstrap.cc:654] T f1890c9d5c664106bacc97d440c9cbc2 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:45.153170 23575 raft_consensus.cc:348] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:45.153280 23575 raft_consensus.cc:374] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:45.153307 23575 raft_consensus.cc:729] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:45.153455 23575 consensus_queue.cc:260] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:45.153667 23575 ts_tablet_manager.cc:1398] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:45.153913 23573 consensus_queue.cc:260] T f5f0690b92c140c49fe10ff46e0136df P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.154366 23578 tablet_bootstrap.cc:492] T f1890c9d5c664106bacc97d440c9cbc2 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:45.154363 23573 ts_tablet_manager.cc:1398] T f5f0690b92c140c49fe10ff46e0136df P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20240502 07:58:45.154440 23578 ts_tablet_manager.cc:1367] T f1890c9d5c664106bacc97d440c9cbc2 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:45.154520 23573 tablet_bootstrap.cc:492] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:45.154815 23578 raft_consensus.cc:348] T f1890c9d5c664106bacc97d440c9cbc2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:45.154919 23578 raft_consensus.cc:374] T f1890c9d5c664106bacc97d440c9cbc2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:45.154949 23578 raft_consensus.cc:729] T f1890c9d5c664106bacc97d440c9cbc2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:45.155069 23578 consensus_queue.cc:260] T f1890c9d5c664106bacc97d440c9cbc2 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:45.155309 23578 ts_tablet_manager.cc:1398] T f1890c9d5c664106bacc97d440c9cbc2 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:45.155395 23573 tablet_bootstrap.cc:654] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:45.156510 23573 tablet_bootstrap.cc:492] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:45.156595 23573 ts_tablet_manager.cc:1367] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:45.157021 23573 raft_consensus.cc:348] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:45.157141 23573 raft_consensus.cc:374] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:45.157174 23573 raft_consensus.cc:729] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:45.157291 23573 consensus_queue.cc:260] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:45.157581 23573 ts_tablet_manager.cc:1398] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:45.283128 23369 raft_consensus.cc:482] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:45.283264 23369 raft_consensus.cc:504] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:45.283690 23369 leader_election.cc:288] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:45.284243 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1890c9d5c664106bacc97d440c9cbc2" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:45.284269 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1890c9d5c664106bacc97d440c9cbc2" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:45.284399 22764 raft_consensus.cc:2454] T f1890c9d5c664106bacc97d440c9cbc2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:45.284459 22633 raft_consensus.cc:2454] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:45.284725 22439 leader_election.cc:302] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:45.284895 23369 raft_consensus.cc:2781] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:45.284967 23369 raft_consensus.cc:482] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:45.285025 23369 raft_consensus.cc:3037] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.286276 23369 raft_consensus.cc:504] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:45.286763 23369 leader_election.cc:288] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:45.287266 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1890c9d5c664106bacc97d440c9cbc2" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:45.287421 22764 raft_consensus.cc:3037] T f1890c9d5c664106bacc97d440c9cbc2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.288450 23482 raft_consensus.cc:482] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:45.288558 23482 raft_consensus.cc:504] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:45.288637 22764 raft_consensus.cc:2454] T f1890c9d5c664106bacc97d440c9cbc2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:58:45.288954 23482 leader_election.cc:288] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:45.289031 22439 leader_election.cc:302] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:45.288951 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1890c9d5c664106bacc97d440c9cbc2" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:45.289141 22633 raft_consensus.cc:3037] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.289192 23369 raft_consensus.cc:2781] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:45.289531 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9c0c75053be41f59786f1cf4fb3062c" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:45.289588 23369 raft_consensus.cc:686] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:58:45.289670 22370 raft_consensus.cc:2454] T d9c0c75053be41f59786f1cf4fb3062c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:45.289739 23369 consensus_queue.cc:237] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:45.289954 22570 leader_election.cc:302] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:45.290117 23482 raft_consensus.cc:2781] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:45.290159 23482 raft_consensus.cc:482] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:45.290191 23482 raft_consensus.cc:3037] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.290297 22633 raft_consensus.cc:2454] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:58:45.291086 23482 raft_consensus.cc:504] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:45.291458 23482 leader_election.cc:288] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:45.292052 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9c0c75053be41f59786f1cf4fb3062c" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:45.292176 22765 raft_consensus.cc:3037] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.292572 22224 catalog_manager.cc:5484] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:58:45.293095 22765 raft_consensus.cc:2454] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:45.293421 22570 leader_election.cc:302] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:45.293625 23482 raft_consensus.cc:2781] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:45.293831 23482 raft_consensus.cc:686] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:45.293931 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9c0c75053be41f59786f1cf4fb3062c" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:45.293965 23482 consensus_queue.cc:237] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:45.294050 22370 raft_consensus.cc:3037] T d9c0c75053be41f59786f1cf4fb3062c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.294979 22370 raft_consensus.cc:2454] T d9c0c75053be41f59786f1cf4fb3062c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:45.296208 22229 catalog_manager.cc:5484] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:58:45.296747 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9c0c75053be41f59786f1cf4fb3062c" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:45.296895 22764 raft_consensus.cc:2362] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1. Re-sending same reply.
I20240502 07:58:45.418551 23550 raft_consensus.cc:482] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:45.418745 23550 raft_consensus.cc:504] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.419778 23550 leader_election.cc:288] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:45.420419 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f0690b92c140c49fe10ff46e0136df" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:45.420419 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f0690b92c140c49fe10ff46e0136df" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:45.420583 22502 raft_consensus.cc:2454] T f5f0690b92c140c49fe10ff46e0136df P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:45.420583 22633 raft_consensus.cc:2454] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:45.420958 22305 leader_election.cc:302] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:45.421154 23550 raft_consensus.cc:2781] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:45.421238 23550 raft_consensus.cc:482] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:45.421308 23550 raft_consensus.cc:3037] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.422669 23550 raft_consensus.cc:504] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.423188 23550 leader_election.cc:288] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:45.423655 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f0690b92c140c49fe10ff46e0136df" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:45.423655 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f0690b92c140c49fe10ff46e0136df" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:45.423810 22633 raft_consensus.cc:3037] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.423810 22502 raft_consensus.cc:3037] T f5f0690b92c140c49fe10ff46e0136df P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.424782 22502 raft_consensus.cc:2454] T f5f0690b92c140c49fe10ff46e0136df P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:45.424782 22633 raft_consensus.cc:2454] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:45.425122 22305 leader_election.cc:302] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:45.425290 23550 raft_consensus.cc:2781] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:45.425413 23550 raft_consensus.cc:686] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:58:45.425767 23550 consensus_queue.cc:237] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.428265 22229 catalog_manager.cc:5484] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
W20240502 07:58:45.560855 23590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:45.561111 23590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:45.564257 23590 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240502 07:58:45.652650 22633 raft_consensus.cc:1261] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:45.653050 22502 raft_consensus.cc:1261] T f5f0690b92c140c49fe10ff46e0136df P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:45.653097 23550 consensus_queue.cc:1035] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:45.653461 23426 consensus_queue.cc:1035] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:45.654464 22764 raft_consensus.cc:1261] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:45.655627 23482 consensus_queue.cc:1035] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:45.658145 22764 raft_consensus.cc:1261] T f1890c9d5c664106bacc97d440c9cbc2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:45.658596 23369 consensus_queue.cc:1035] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:45.659986 22370 raft_consensus.cc:1261] T d9c0c75053be41f59786f1cf4fb3062c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:45.660385 23587 consensus_queue.cc:1035] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:45.660837 22634 raft_consensus.cc:1261] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:45.661389 23369 consensus_queue.cc:1035] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 113.34 ms
time per row: 0.0566702 ms
I20240502 07:58:45.721714 22229 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:45.727836 22613 tablet_service.cc:1467] Processing CreateTablet for tablet 1ec8ca8bbe2341dc8000de45ef289415 (DEFAULT_TABLE table=rebalance_test_table_4 [id=74c99be4bec647c09e6275ef44e45334]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:45.728128 22744 tablet_service.cc:1467] Processing CreateTablet for tablet 58c28df209214cb6a15d41423a5cbd32 (DEFAULT_TABLE table=rebalance_test_table_4 [id=74c99be4bec647c09e6275ef44e45334]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:45.728200 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ec8ca8bbe2341dc8000de45ef289415. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:45.728405 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58c28df209214cb6a15d41423a5cbd32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:45.728541 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 58c28df209214cb6a15d41423a5cbd32 (DEFAULT_TABLE table=rebalance_test_table_4 [id=74c99be4bec647c09e6275ef44e45334]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:45.728794 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58c28df209214cb6a15d41423a5cbd32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:45.728974 22482 tablet_service.cc:1467] Processing CreateTablet for tablet 1ec8ca8bbe2341dc8000de45ef289415 (DEFAULT_TABLE table=rebalance_test_table_4 [id=74c99be4bec647c09e6275ef44e45334]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:45.729246 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ec8ca8bbe2341dc8000de45ef289415. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:45.729602 22351 tablet_service.cc:1467] Processing CreateTablet for tablet 1ec8ca8bbe2341dc8000de45ef289415 (DEFAULT_TABLE table=rebalance_test_table_4 [id=74c99be4bec647c09e6275ef44e45334]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:45.729806 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ec8ca8bbe2341dc8000de45ef289415. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:45.731459 22612 tablet_service.cc:1467] Processing CreateTablet for tablet 696f4ce96be64070a1aa2760df43720b (DEFAULT_TABLE table=rebalance_test_table_4 [id=74c99be4bec647c09e6275ef44e45334]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:45.731554 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 58c28df209214cb6a15d41423a5cbd32 (DEFAULT_TABLE table=rebalance_test_table_4 [id=74c99be4bec647c09e6275ef44e45334]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:45.731768 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58c28df209214cb6a15d41423a5cbd32. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:45.732085 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 696f4ce96be64070a1aa2760df43720b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:45.733299 22351 tablet_service.cc:1467] Processing CreateTablet for tablet 696f4ce96be64070a1aa2760df43720b (DEFAULT_TABLE table=rebalance_test_table_4 [id=74c99be4bec647c09e6275ef44e45334]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:45.733664 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 696f4ce96be64070a1aa2760df43720b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:45.734239 22744 tablet_service.cc:1467] Processing CreateTablet for tablet 696f4ce96be64070a1aa2760df43720b (DEFAULT_TABLE table=rebalance_test_table_4 [id=74c99be4bec647c09e6275ef44e45334]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:45.734475 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 696f4ce96be64070a1aa2760df43720b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:45.734658 23653 tablet_bootstrap.cc:492] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:45.735560 23653 tablet_bootstrap.cc:654] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:45.735826 23655 tablet_bootstrap.cc:492] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:45.736848 23655 tablet_bootstrap.cc:654] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:45.736861 23654 tablet_bootstrap.cc:492] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:45.737092 23656 tablet_bootstrap.cc:492] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:45.737634 23654 tablet_bootstrap.cc:654] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:45.738093 23655 tablet_bootstrap.cc:492] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:45.738101 23656 tablet_bootstrap.cc:654] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:45.738181 23655 ts_tablet_manager.cc:1367] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:45.738646 23654 tablet_bootstrap.cc:492] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:45.738711 23654 ts_tablet_manager.cc:1367] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:45.738654 23655 raft_consensus.cc:348] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.738765 23655 raft_consensus.cc:374] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:45.738808 23655 raft_consensus.cc:729] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:45.738945 23655 consensus_queue.cc:260] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.739104 23654 raft_consensus.cc:348] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:45.739195 23654 raft_consensus.cc:374] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:45.739223 23654 raft_consensus.cc:729] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:45.739244 23655 ts_tablet_manager.cc:1398] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:45.739326 23654 consensus_queue.cc:260] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:45.739419 23655 tablet_bootstrap.cc:492] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:45.739543 23654 ts_tablet_manager.cc:1398] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:45.739692 23654 tablet_bootstrap.cc:492] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:45.740165 23656 tablet_bootstrap.cc:492] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:45.740267 23656 ts_tablet_manager.cc:1367] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:45.740283 23655 tablet_bootstrap.cc:654] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:45.740523 23654 tablet_bootstrap.cc:654] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:45.740743 23656 raft_consensus.cc:348] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.740849 23656 raft_consensus.cc:374] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:45.740881 23656 raft_consensus.cc:729] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:45.740993 23656 consensus_queue.cc:260] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.741253 23656 ts_tablet_manager.cc:1398] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:45.741446 23656 tablet_bootstrap.cc:492] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:45.741724 23654 tablet_bootstrap.cc:492] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:45.741819 23654 ts_tablet_manager.cc:1367] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:45.742259 23654 raft_consensus.cc:348] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.742353 23654 raft_consensus.cc:374] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:45.742385 23654 raft_consensus.cc:729] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:45.742493 23654 consensus_queue.cc:260] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.742784 23654 ts_tablet_manager.cc:1398] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:45.742832 23656 tablet_bootstrap.cc:654] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:45.742930 23654 tablet_bootstrap.cc:492] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:45.743427 23655 tablet_bootstrap.cc:492] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:45.743501 23655 ts_tablet_manager.cc:1367] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:45.743781 23654 tablet_bootstrap.cc:654] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:45.744143 23653 tablet_bootstrap.cc:492] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:45.744088 23655 raft_consensus.cc:348] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:45.744243 23655 raft_consensus.cc:374] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:45.744292 23655 raft_consensus.cc:729] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:45.744307 23653 ts_tablet_manager.cc:1367] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 07:58:45.744544 23655 consensus_queue.cc:260] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:45.744966 23655 ts_tablet_manager.cc:1398] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:58:45.745064 23653 raft_consensus.cc:348] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:45.745165 23653 raft_consensus.cc:374] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:45.745198 23653 raft_consensus.cc:729] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:45.745364 23653 consensus_queue.cc:260] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:45.745762 23653 ts_tablet_manager.cc:1398] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:58:45.745994 23653 tablet_bootstrap.cc:492] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:45.746042 23654 tablet_bootstrap.cc:492] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:45.746127 23654 ts_tablet_manager.cc:1367] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20240502 07:58:45.746555 23654 raft_consensus.cc:348] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.746649 23654 raft_consensus.cc:374] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:45.746678 23654 raft_consensus.cc:729] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:45.746779 23654 consensus_queue.cc:260] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.746927 23653 tablet_bootstrap.cc:654] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:45.747079 23654 ts_tablet_manager.cc:1398] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:45.748009 23656 tablet_bootstrap.cc:492] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:45.748109 23656 ts_tablet_manager.cc:1367] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 07:58:45.748576 23656 raft_consensus.cc:348] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.748749 23656 raft_consensus.cc:374] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:45.748832 23656 raft_consensus.cc:729] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:45.749018 23653 tablet_bootstrap.cc:492] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:45.749006 23656 consensus_queue.cc:260] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.749130 23653 ts_tablet_manager.cc:1367] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:45.749460 23656 ts_tablet_manager.cc:1398] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:45.749678 23653 raft_consensus.cc:348] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.749986 23653 raft_consensus.cc:374] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:45.750046 23653 raft_consensus.cc:729] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:45.750181 23653 consensus_queue.cc:260] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.750494 23653 ts_tablet_manager.cc:1398] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:45.822680 23431 raft_consensus.cc:482] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:45.822804 23431 raft_consensus.cc:504] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.823235 23431 leader_election.cc:288] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:45.823730 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "696f4ce96be64070a1aa2760df43720b" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:45.823801 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "696f4ce96be64070a1aa2760df43720b" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:45.823877 22370 raft_consensus.cc:2454] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:45.824002 22633 raft_consensus.cc:2454] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:45.824201 22701 leader_election.cc:302] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:45.824362 23431 raft_consensus.cc:2781] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:45.824421 23431 raft_consensus.cc:482] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:45.824458 23431 raft_consensus.cc:3037] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.825238 23431 raft_consensus.cc:504] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.825593 23431 leader_election.cc:288] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:45.825980 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "696f4ce96be64070a1aa2760df43720b" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:45.826022 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "696f4ce96be64070a1aa2760df43720b" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:45.826082 22370 raft_consensus.cc:3037] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.826148 22633 raft_consensus.cc:3037] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.826840 22370 raft_consensus.cc:2454] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:45.826933 22633 raft_consensus.cc:2454] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:45.827101 22701 leader_election.cc:302] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:45.827303 23431 raft_consensus.cc:2781] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:45.827490 23431 raft_consensus.cc:686] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:58:45.827637 23431 consensus_queue.cc:237] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.829476 22224 catalog_manager.cc:5484] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:58:45.914531 23426 raft_consensus.cc:482] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:45.914672 23426 raft_consensus.cc:504] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:45.915134 23426 leader_election.cc:288] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:45.915596 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58c28df209214cb6a15d41423a5cbd32" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:45.915607 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58c28df209214cb6a15d41423a5cbd32" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:45.915741 22502 raft_consensus.cc:2454] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:45.915741 22764 raft_consensus.cc:2454] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:45.916071 22308 leader_election.cc:302] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:45.916280 23426 raft_consensus.cc:2781] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:45.916342 23426 raft_consensus.cc:482] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:45.916368 23426 raft_consensus.cc:3037] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.917197 23426 raft_consensus.cc:504] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:45.917536 23426 leader_election.cc:288] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:45.917892 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58c28df209214cb6a15d41423a5cbd32" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:45.917896 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58c28df209214cb6a15d41423a5cbd32" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:45.918006 22764 raft_consensus.cc:3037] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.918032 22502 raft_consensus.cc:3037] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.918721 22502 raft_consensus.cc:2454] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:45.918722 22764 raft_consensus.cc:2454] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:45.919010 22305 leader_election.cc:302] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:45.919195 23426 raft_consensus.cc:2781] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:45.919299 23426 raft_consensus.cc:686] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:58:45.919409 23426 consensus_queue.cc:237] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:45.921151 22224 catalog_manager.cc:5484] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:58:45.961621 23587 raft_consensus.cc:482] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:45.961774 23587 raft_consensus.cc:504] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.962167 23587 leader_election.cc:288] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:45.962683 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec8ca8bbe2341dc8000de45ef289415" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:45.962715 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec8ca8bbe2341dc8000de45ef289415" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:45.962819 22370 raft_consensus.cc:2454] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:45.962853 22502 raft_consensus.cc:2454] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:45.963173 22567 leader_election.cc:302] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:45.963358 23587 raft_consensus.cc:2781] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:45.963420 23587 raft_consensus.cc:482] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:45.963469 23587 raft_consensus.cc:3037] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.964349 23587 raft_consensus.cc:504] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.964690 23587 leader_election.cc:288] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:45.965272 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec8ca8bbe2341dc8000de45ef289415" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:45.965291 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ec8ca8bbe2341dc8000de45ef289415" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:45.965404 22370 raft_consensus.cc:3037] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.965435 22502 raft_consensus.cc:3037] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:45.966120 22370 raft_consensus.cc:2454] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:45.966172 22502 raft_consensus.cc:2454] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:45.966414 22570 leader_election.cc:302] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:45.966573 23587 raft_consensus.cc:2781] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:45.966660 23587 raft_consensus.cc:686] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:45.966804 23587 consensus_queue.cc:237] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:45.968621 22224 catalog_manager.cc:5484] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
W20240502 07:58:46.082698 23670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:46.083058 23670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:46.087302 23670 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240502 07:58:46.188233 22764 raft_consensus.cc:1261] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:46.188233 22502 raft_consensus.cc:1261] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:46.188827 23426 consensus_queue.cc:1035] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:46.190197 22633 raft_consensus.cc:1261] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:46.190577 22370 raft_consensus.cc:1261] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:46.191102 22502 raft_consensus.cc:1261] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:46.191593 22370 raft_consensus.cc:1261] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:46.191833 23666 consensus_queue.cc:1035] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:46.192118 23431 consensus_queue.cc:1035] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:46.192510 23587 consensus_queue.cc:1035] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:46.193212 23550 consensus_queue.cc:1035] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:46.193464 23482 consensus_queue.cc:1035] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:46.221578 23710 mvcc.cc:205] Tried to move back new op lower bound from 7023152030480973824 to 7023152029367349248. Current Snapshot: MvccSnapshot[applied={T|T < 7023152030462332928}]
INSERT report
rows total: 2000
time total: 125.511 ms
time per row: 0.0627557 ms
I20240502 07:58:46.260413 22228 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:46.266186 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 4e202e9cf0154eb9814a7e3ad93f1e42 (DEFAULT_TABLE table=rebalance_test_table_5 [id=80b9c0f932754699a4f41d2c2188b70b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:46.266252 22744 tablet_service.cc:1467] Processing CreateTablet for tablet 4e202e9cf0154eb9814a7e3ad93f1e42 (DEFAULT_TABLE table=rebalance_test_table_5 [id=80b9c0f932754699a4f41d2c2188b70b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:46.266541 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e202e9cf0154eb9814a7e3ad93f1e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.266549 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e202e9cf0154eb9814a7e3ad93f1e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.266579 22612 tablet_service.cc:1467] Processing CreateTablet for tablet 4e202e9cf0154eb9814a7e3ad93f1e42 (DEFAULT_TABLE table=rebalance_test_table_5 [id=80b9c0f932754699a4f41d2c2188b70b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:46.266831 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e202e9cf0154eb9814a7e3ad93f1e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.267165 22613 tablet_service.cc:1467] Processing CreateTablet for tablet 648a25ebebe44061bca2844ec5b048c5 (DEFAULT_TABLE table=rebalance_test_table_5 [id=80b9c0f932754699a4f41d2c2188b70b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:46.267385 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 648a25ebebe44061bca2844ec5b048c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.269647 23729 tablet_bootstrap.cc:492] T 4e202e9cf0154eb9814a7e3ad93f1e42 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:46.269991 22744 tablet_service.cc:1467] Processing CreateTablet for tablet 648a25ebebe44061bca2844ec5b048c5 (DEFAULT_TABLE table=rebalance_test_table_5 [id=80b9c0f932754699a4f41d2c2188b70b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:46.270249 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 648a25ebebe44061bca2844ec5b048c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.270361 22351 tablet_service.cc:1467] Processing CreateTablet for tablet dbf60f171dfe4c88840587bf0df64cac (DEFAULT_TABLE table=rebalance_test_table_5 [id=80b9c0f932754699a4f41d2c2188b70b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:46.270629 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf60f171dfe4c88840587bf0df64cac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.270634 23729 tablet_bootstrap.cc:654] T 4e202e9cf0154eb9814a7e3ad93f1e42 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:46.270699 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 648a25ebebe44061bca2844ec5b048c5 (DEFAULT_TABLE table=rebalance_test_table_5 [id=80b9c0f932754699a4f41d2c2188b70b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:46.270920 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 648a25ebebe44061bca2844ec5b048c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.272359 23731 tablet_bootstrap.cc:492] T dbf60f171dfe4c88840587bf0df64cac P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:46.272534 23728 tablet_bootstrap.cc:492] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:46.273066 22482 tablet_service.cc:1467] Processing CreateTablet for tablet dbf60f171dfe4c88840587bf0df64cac (DEFAULT_TABLE table=rebalance_test_table_5 [id=80b9c0f932754699a4f41d2c2188b70b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:46.273240 23731 tablet_bootstrap.cc:654] T dbf60f171dfe4c88840587bf0df64cac P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:46.273306 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf60f171dfe4c88840587bf0df64cac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.273488 23728 tablet_bootstrap.cc:654] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:46.274114 23727 tablet_bootstrap.cc:492] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:46.274432 23729 tablet_bootstrap.cc:492] T 4e202e9cf0154eb9814a7e3ad93f1e42 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:46.274518 23729 ts_tablet_manager.cc:1367] T 4e202e9cf0154eb9814a7e3ad93f1e42 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:46.274948 23729 raft_consensus.cc:348] T 4e202e9cf0154eb9814a7e3ad93f1e42 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:46.275040 23727 tablet_bootstrap.cc:654] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:46.275049 23729 raft_consensus.cc:374] T 4e202e9cf0154eb9814a7e3ad93f1e42 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:46.275081 23729 raft_consensus.cc:729] T 4e202e9cf0154eb9814a7e3ad93f1e42 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:46.275169 22612 tablet_service.cc:1467] Processing CreateTablet for tablet dbf60f171dfe4c88840587bf0df64cac (DEFAULT_TABLE table=rebalance_test_table_5 [id=80b9c0f932754699a4f41d2c2188b70b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:46.275202 23729 consensus_queue.cc:260] T 4e202e9cf0154eb9814a7e3ad93f1e42 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:46.275399 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf60f171dfe4c88840587bf0df64cac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.275497 23729 ts_tablet_manager.cc:1398] T 4e202e9cf0154eb9814a7e3ad93f1e42 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:46.275667 23729 tablet_bootstrap.cc:492] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:46.275871 23731 tablet_bootstrap.cc:492] T dbf60f171dfe4c88840587bf0df64cac P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:46.275950 23731 ts_tablet_manager.cc:1367] T dbf60f171dfe4c88840587bf0df64cac P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 07:58:46.276427 23731 raft_consensus.cc:348] T dbf60f171dfe4c88840587bf0df64cac P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:46.276531 23731 raft_consensus.cc:374] T dbf60f171dfe4c88840587bf0df64cac P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:46.276552 23729 tablet_bootstrap.cc:654] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:46.276563 23731 raft_consensus.cc:729] T dbf60f171dfe4c88840587bf0df64cac P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:46.277670 23729 tablet_bootstrap.cc:492] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:46.277757 23729 ts_tablet_manager.cc:1367] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:46.277791 23731 consensus_queue.cc:260] T dbf60f171dfe4c88840587bf0df64cac P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:46.278198 23729 raft_consensus.cc:348] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:46.278299 23729 raft_consensus.cc:374] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:46.278332 23729 raft_consensus.cc:729] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:46.278434 23729 consensus_queue.cc:260] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:46.278681 23729 ts_tablet_manager.cc:1398] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:46.278798 23727 tablet_bootstrap.cc:492] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:46.278829 23729 tablet_bootstrap.cc:492] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:46.278903 23727 ts_tablet_manager.cc:1367] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:46.278973 23731 ts_tablet_manager.cc:1398] T dbf60f171dfe4c88840587bf0df64cac P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 07:58:46.279395 23727 raft_consensus.cc:348] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:46.279500 23727 raft_consensus.cc:374] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:46.279532 23727 raft_consensus.cc:729] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:46.279660 23728 tablet_bootstrap.cc:492] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:46.279700 23729 tablet_bootstrap.cc:654] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:46.279661 23727 consensus_queue.cc:260] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:46.279757 23728 ts_tablet_manager.cc:1367] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:58:46.279944 23727 ts_tablet_manager.cc:1398] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:46.280100 23727 tablet_bootstrap.cc:492] T 648a25ebebe44061bca2844ec5b048c5 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:46.280213 23728 raft_consensus.cc:348] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:46.280304 23728 raft_consensus.cc:374] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:46.280339 23728 raft_consensus.cc:729] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:46.280457 23728 consensus_queue.cc:260] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:46.280718 23728 ts_tablet_manager.cc:1398] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:46.280838 23729 tablet_bootstrap.cc:492] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:46.280881 23728 tablet_bootstrap.cc:492] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:46.280910 23729 ts_tablet_manager.cc:1367] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:58:46.280970 23727 tablet_bootstrap.cc:654] T 648a25ebebe44061bca2844ec5b048c5 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:46.281359 23729 raft_consensus.cc:348] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:46.281481 23729 raft_consensus.cc:374] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:46.281533 23729 raft_consensus.cc:729] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:46.281643 23729 consensus_queue.cc:260] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:46.281839 23728 tablet_bootstrap.cc:654] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:46.281953 23729 ts_tablet_manager.cc:1398] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:46.282155 23727 tablet_bootstrap.cc:492] T 648a25ebebe44061bca2844ec5b048c5 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:46.282241 23727 ts_tablet_manager.cc:1367] T 648a25ebebe44061bca2844ec5b048c5 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:46.282696 23727 raft_consensus.cc:348] T 648a25ebebe44061bca2844ec5b048c5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:46.282792 23727 raft_consensus.cc:374] T 648a25ebebe44061bca2844ec5b048c5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:46.282830 23727 raft_consensus.cc:729] T 648a25ebebe44061bca2844ec5b048c5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:46.282953 23727 consensus_queue.cc:260] T 648a25ebebe44061bca2844ec5b048c5 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:46.283227 23728 tablet_bootstrap.cc:492] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:46.283236 23727 ts_tablet_manager.cc:1398] T 648a25ebebe44061bca2844ec5b048c5 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:46.283303 23728 ts_tablet_manager.cc:1367] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:46.283702 23728 raft_consensus.cc:348] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:46.283803 23728 raft_consensus.cc:374] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:46.283835 23728 raft_consensus.cc:729] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:46.283951 23728 consensus_queue.cc:260] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:46.284233 23728 ts_tablet_manager.cc:1398] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:46.284373 23728 tablet_bootstrap.cc:492] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:46.285259 23728 tablet_bootstrap.cc:654] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:46.286360 23728 tablet_bootstrap.cc:492] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:46.286440 23728 ts_tablet_manager.cc:1367] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:46.286913 23728 raft_consensus.cc:348] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:46.287001 23728 raft_consensus.cc:374] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:46.287036 23728 raft_consensus.cc:729] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:46.287158 23728 consensus_queue.cc:260] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:46.287426 23728 ts_tablet_manager.cc:1398] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:46.424785 23379 raft_consensus.cc:482] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:46.424930 23379 raft_consensus.cc:504] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:46.425411 23379 leader_election.cc:288] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:46.425938 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf60f171dfe4c88840587bf0df64cac" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:46.425961 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf60f171dfe4c88840587bf0df64cac" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:46.426092 22633 raft_consensus.cc:2454] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:46.426092 22370 raft_consensus.cc:2454] T dbf60f171dfe4c88840587bf0df64cac P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:46.426399 22439 leader_election.cc:302] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:46.426563 23379 raft_consensus.cc:2781] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:46.426625 23379 raft_consensus.cc:482] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:46.426671 23379 raft_consensus.cc:3037] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:46.427541 23379 raft_consensus.cc:504] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:46.427853 23379 leader_election.cc:288] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:46.428289 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf60f171dfe4c88840587bf0df64cac" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:46.428289 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf60f171dfe4c88840587bf0df64cac" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:46.428412 22370 raft_consensus.cc:3037] T dbf60f171dfe4c88840587bf0df64cac P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:46.428412 22633 raft_consensus.cc:3037] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:46.429388 22370 raft_consensus.cc:2454] T dbf60f171dfe4c88840587bf0df64cac P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:58:46.429388 22633 raft_consensus.cc:2454] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:58:46.429662 22439 leader_election.cc:302] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:46.429914 23379 raft_consensus.cc:2781] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:46.430126 23379 raft_consensus.cc:686] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:58:46.430274 23379 consensus_queue.cc:237] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:46.431835 22222 catalog_manager.cc:5484] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:58:46.525563 23726 raft_consensus.cc:482] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:46.525741 23726 raft_consensus.cc:504] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:46.526252 23726 leader_election.cc:288] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:46.526684 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "648a25ebebe44061bca2844ec5b048c5" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:46.526686 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "648a25ebebe44061bca2844ec5b048c5" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:46.526834 22502 raft_consensus.cc:2454] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:46.526839 22764 raft_consensus.cc:2454] T 648a25ebebe44061bca2844ec5b048c5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:46.527092 22567 leader_election.cc:302] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:46.527283 23726 raft_consensus.cc:2781] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:46.527356 23726 raft_consensus.cc:482] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:46.527387 23726 raft_consensus.cc:3037] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:46.528353 23726 raft_consensus.cc:504] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:46.528760 23726 leader_election.cc:288] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:46.529165 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "648a25ebebe44061bca2844ec5b048c5" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:46.529165 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "648a25ebebe44061bca2844ec5b048c5" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:46.529307 22502 raft_consensus.cc:3037] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:46.529307 22764 raft_consensus.cc:3037] T 648a25ebebe44061bca2844ec5b048c5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:46.530306 22764 raft_consensus.cc:2454] T 648a25ebebe44061bca2844ec5b048c5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:46.530306 22502 raft_consensus.cc:2454] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:46.530642 22567 leader_election.cc:302] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:46.530831 23726 raft_consensus.cc:2781] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:46.530920 23726 raft_consensus.cc:686] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:46.531092 23726 consensus_queue.cc:237] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:46.532871 22222 catalog_manager.cc:5484] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:58:46.615976 23666 raft_consensus.cc:482] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:46.616118 23666 raft_consensus.cc:504] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:46.616514 23666 leader_election.cc:288] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:46.617017 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e202e9cf0154eb9814a7e3ad93f1e42" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:46.617017 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e202e9cf0154eb9814a7e3ad93f1e42" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:46.617178 22633 raft_consensus.cc:2454] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:46.617178 22502 raft_consensus.cc:2454] T 4e202e9cf0154eb9814a7e3ad93f1e42 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:46.617520 22701 leader_election.cc:302] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:46.617718 23666 raft_consensus.cc:2781] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:46.617785 23666 raft_consensus.cc:482] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:46.617837 23666 raft_consensus.cc:3037] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:46.618772 23666 raft_consensus.cc:504] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:46.619084 23666 leader_election.cc:288] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:46.619397 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e202e9cf0154eb9814a7e3ad93f1e42" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:46.619431 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e202e9cf0154eb9814a7e3ad93f1e42" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:46.619509 22502 raft_consensus.cc:3037] T 4e202e9cf0154eb9814a7e3ad93f1e42 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:46.619529 22633 raft_consensus.cc:3037] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:46.620209 22502 raft_consensus.cc:2454] T 4e202e9cf0154eb9814a7e3ad93f1e42 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:46.620465 22698 leader_election.cc:302] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:46.620616 23666 raft_consensus.cc:2781] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:46.620710 23666 raft_consensus.cc:686] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:58:46.620862 23666 consensus_queue.cc:237] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:46.622524 22222 catalog_manager.cc:5484] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:58:46.634056 22633 raft_consensus.cc:2454] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
W20240502 07:58:46.753082 23744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:46.753369 23744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:46.756390 23744 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240502 07:58:46.821734 23379 consensus_queue.cc:1035] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:46.826011 23379 consensus_queue.cc:1035] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:46.845788 22634 raft_consensus.cc:1261] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:46.849828 23431 consensus_queue.cc:1035] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:46.850698 22502 raft_consensus.cc:1261] T 4e202e9cf0154eb9814a7e3ad93f1e42 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:46.851758 23666 consensus_queue.cc:1035] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:46.856138 22501 raft_consensus.cc:1261] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:46.856727 22764 raft_consensus.cc:1261] T 648a25ebebe44061bca2844ec5b048c5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:46.857142 23726 consensus_queue.cc:1035] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:46.858867 23726 consensus_queue.cc:1035] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:46.874604 23785 mvcc.cc:205] Tried to move back new op lower bound from 7023152033155792896 to 7023152032240496640. Current Snapshot: MvccSnapshot[applied={T|T < 7023152033155792896}]
I20240502 07:58:46.880131 23780 mvcc.cc:205] Tried to move back new op lower bound from 7023152033191694336 to 7023152031872688128. Current Snapshot: MvccSnapshot[applied={T|T < 7023152033191694336}]
INSERT report
rows total: 2000
time total: 113.931 ms
time per row: 0.0569654 ms
I20240502 07:58:46.912961 22228 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:46.918388 22482 tablet_service.cc:1467] Processing CreateTablet for tablet e6a499fd83554a2c8e3e837aedddb61b (DEFAULT_TABLE table=rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:46.918545 22744 tablet_service.cc:1467] Processing CreateTablet for tablet e6a499fd83554a2c8e3e837aedddb61b (DEFAULT_TABLE table=rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:46.918712 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6a499fd83554a2c8e3e837aedddb61b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.918798 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6a499fd83554a2c8e3e837aedddb61b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.919432 22745 tablet_service.cc:1467] Processing CreateTablet for tablet b420738e4ec14fe6bc9b9adddb86ec46 (DEFAULT_TABLE table=rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:46.919657 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b420738e4ec14fe6bc9b9adddb86ec46. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.921090 22743 tablet_service.cc:1467] Processing CreateTablet for tablet b9557af8565d4588b119f1915e76c3dc (DEFAULT_TABLE table=rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:46.921343 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9557af8565d4588b119f1915e76c3dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.921438 22352 tablet_service.cc:1467] Processing CreateTablet for tablet b420738e4ec14fe6bc9b9adddb86ec46 (DEFAULT_TABLE table=rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:46.921468 22351 tablet_service.cc:1467] Processing CreateTablet for tablet e6a499fd83554a2c8e3e837aedddb61b (DEFAULT_TABLE table=rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:46.921700 22350 tablet_service.cc:1467] Processing CreateTablet for tablet b9557af8565d4588b119f1915e76c3dc (DEFAULT_TABLE table=rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:46.921988 22612 tablet_service.cc:1467] Processing CreateTablet for tablet b420738e4ec14fe6bc9b9adddb86ec46 (DEFAULT_TABLE table=rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:46.921711 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b420738e4ec14fe6bc9b9adddb86ec46. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.922240 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b420738e4ec14fe6bc9b9adddb86ec46. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.922379 23803 tablet_bootstrap.cc:492] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:46.923199 23803 tablet_bootstrap.cc:654] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:46.923879 23802 tablet_bootstrap.cc:492] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:46.924271 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6a499fd83554a2c8e3e837aedddb61b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.924329 23803 tablet_bootstrap.cc:492] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:46.924439 23803 ts_tablet_manager.cc:1367] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:46.924772 23802 tablet_bootstrap.cc:654] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:46.925683 22350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9557af8565d4588b119f1915e76c3dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.925729 23803 raft_consensus.cc:348] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:46.925849 23803 raft_consensus.cc:374] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:46.925884 23803 raft_consensus.cc:729] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:46.926036 23803 consensus_queue.cc:260] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:46.926321 23803 ts_tablet_manager.cc:1398] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:46.926928 22482 tablet_service.cc:1467] Processing CreateTablet for tablet b9557af8565d4588b119f1915e76c3dc (DEFAULT_TABLE table=rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:46.927178 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9557af8565d4588b119f1915e76c3dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:46.928076 23806 tablet_bootstrap.cc:492] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:46.928725 23806 tablet_bootstrap.cc:654] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:46.929116 23803 tablet_bootstrap.cc:492] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:46.929292 23805 tablet_bootstrap.cc:492] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:46.929957 23806 tablet_bootstrap.cc:492] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:46.930099 23805 tablet_bootstrap.cc:654] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:46.930105 23806 ts_tablet_manager.cc:1367] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:46.930649 23806 raft_consensus.cc:348] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:46.930814 23806 raft_consensus.cc:374] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:46.930891 23806 raft_consensus.cc:729] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:46.931120 23806 consensus_queue.cc:260] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:46.931279 23802 tablet_bootstrap.cc:492] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:46.931289 23803 tablet_bootstrap.cc:654] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:46.931367 23802 ts_tablet_manager.cc:1367] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:58:46.931721 23802 raft_consensus.cc:348] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:46.931809 23802 raft_consensus.cc:374] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:46.931844 23802 raft_consensus.cc:729] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:46.931972 23802 consensus_queue.cc:260] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:46.932278 23802 ts_tablet_manager.cc:1398] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:46.932456 23802 tablet_bootstrap.cc:492] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:46.932531 23805 tablet_bootstrap.cc:492] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:46.932649 23805 ts_tablet_manager.cc:1367] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:46.932650 23803 tablet_bootstrap.cc:492] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:46.932720 23803 ts_tablet_manager.cc:1367] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:46.933151 23803 raft_consensus.cc:348] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:46.933225 23806 ts_tablet_manager.cc:1398] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:46.933147 23805 raft_consensus.cc:348] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:46.933246 23803 raft_consensus.cc:374] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:46.933269 23805 raft_consensus.cc:374] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:46.933276 23803 raft_consensus.cc:729] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:46.933300 23802 tablet_bootstrap.cc:654] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:46.933306 23805 raft_consensus.cc:729] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:46.933470 23805 consensus_queue.cc:260] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:46.933804 23805 ts_tablet_manager.cc:1398] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:46.933563 23803 consensus_queue.cc:260] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:46.934896 23806 tablet_bootstrap.cc:492] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:46.935057 23802 tablet_bootstrap.cc:492] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:46.935142 23802 ts_tablet_manager.cc:1367] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:46.935593 23802 raft_consensus.cc:348] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:46.935698 23802 raft_consensus.cc:374] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:46.935734 23802 raft_consensus.cc:729] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:46.935846 23802 consensus_queue.cc:260] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:46.936092 23802 ts_tablet_manager.cc:1398] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:46.936257 23802 tablet_bootstrap.cc:492] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:46.936369 23803 ts_tablet_manager.cc:1398] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:46.937058 23806 tablet_bootstrap.cc:654] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:46.937762 23802 tablet_bootstrap.cc:654] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:46.938882 23802 tablet_bootstrap.cc:492] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:46.938977 23802 ts_tablet_manager.cc:1367] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:46.939065 23806 tablet_bootstrap.cc:492] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:46.939149 23806 ts_tablet_manager.cc:1367] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:46.939392 23802 raft_consensus.cc:348] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:46.939497 23802 raft_consensus.cc:374] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:46.939534 23802 raft_consensus.cc:729] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:46.939522 23806 raft_consensus.cc:348] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:46.939615 23806 raft_consensus.cc:374] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:46.939646 23806 raft_consensus.cc:729] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:46.939662 23802 consensus_queue.cc:260] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:46.939752 23806 consensus_queue.cc:260] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:46.940037 23806 ts_tablet_manager.cc:1398] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:46.940057 23802 ts_tablet_manager.cc:1398] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:46.940297 23806 tablet_bootstrap.cc:492] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:46.941222 23806 tablet_bootstrap.cc:654] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:46.942548 23806 tablet_bootstrap.cc:492] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:46.942633 23806 ts_tablet_manager.cc:1367] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:46.943272 23806 raft_consensus.cc:348] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:46.943360 23806 raft_consensus.cc:374] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:46.943382 23806 raft_consensus.cc:729] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:46.943467 23806 consensus_queue.cc:260] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:46.943683 23806 ts_tablet_manager.cc:1398] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:46.948108 23773 raft_consensus.cc:482] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:46.948199 23773 raft_consensus.cc:504] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:46.948516 23773 leader_election.cc:288] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:46.948827 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6a499fd83554a2c8e3e837aedddb61b" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:46.948851 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6a499fd83554a2c8e3e837aedddb61b" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:46.948959 22764 raft_consensus.cc:2454] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:46.948966 22502 raft_consensus.cc:2454] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:46.949205 22308 leader_election.cc:302] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:46.949395 23773 raft_consensus.cc:2781] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:46.949446 23773 raft_consensus.cc:482] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:46.949473 23773 raft_consensus.cc:3037] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:46.950277 23773 raft_consensus.cc:504] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:46.950589 23773 leader_election.cc:288] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:46.950932 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6a499fd83554a2c8e3e837aedddb61b" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:46.951027 22502 raft_consensus.cc:3037] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:46.951004 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6a499fd83554a2c8e3e837aedddb61b" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:46.951141 22764 raft_consensus.cc:3037] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:46.951750 22502 raft_consensus.cc:2454] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:46.951889 22764 raft_consensus.cc:2454] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:46.952004 22305 leader_election.cc:302] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:46.952160 23773 raft_consensus.cc:2781] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:46.952253 23773 raft_consensus.cc:686] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:58:46.952384 23773 consensus_queue.cc:237] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:46.954064 22227 catalog_manager.cc:5484] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:58:47.027963 23773 raft_consensus.cc:482] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:47.028183 23666 raft_consensus.cc:482] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:47.028091 23773 raft_consensus.cc:504] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.028282 23666 raft_consensus.cc:504] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.028630 23773 leader_election.cc:288] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:47.028630 23666 leader_election.cc:288] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:47.028996 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9557af8565d4588b119f1915e76c3dc" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:47.029003 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9557af8565d4588b119f1915e76c3dc" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:47.029119 22370 raft_consensus.cc:2454] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:47.029099 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9557af8565d4588b119f1915e76c3dc" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:47.029141 22764 raft_consensus.cc:2454] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:47.029232 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9557af8565d4588b119f1915e76c3dc" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:47.029232 22502 raft_consensus.cc:2454] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:47.029553 22701 leader_election.cc:302] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:47.029718 23666 raft_consensus.cc:2781] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:47.029723 22305 leader_election.cc:302] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:47.029768 23666 raft_consensus.cc:482] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:47.029808 23666 raft_consensus.cc:3037] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:47.029856 23773 raft_consensus.cc:2781] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:47.029924 23773 raft_consensus.cc:482] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:47.029976 23773 raft_consensus.cc:3037] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:47.030714 23666 raft_consensus.cc:504] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.030788 23773 raft_consensus.cc:504] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.031105 23666 leader_election.cc:288] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:47.031304 23773 leader_election.cc:288] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:47.031384 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9557af8565d4588b119f1915e76c3dc" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:47.031421 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9557af8565d4588b119f1915e76c3dc" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:47.031517 22501 raft_consensus.cc:3037] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:47.031533 22370 raft_consensus.cc:2379] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f00f3a1876444ad2bf9c919068ab8014 in current term 1: Already voted for candidate b533a7d8d97943609c6e28e3e36b78a4 in this term.
I20240502 07:58:47.031898 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9557af8565d4588b119f1915e76c3dc" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:47.031941 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9557af8565d4588b119f1915e76c3dc" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:47.032050 22764 raft_consensus.cc:2379] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b533a7d8d97943609c6e28e3e36b78a4 in current term 1: Already voted for candidate f00f3a1876444ad2bf9c919068ab8014 in this term.
I20240502 07:58:47.032330 22308 leader_election.cc:302] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4; no voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c
I20240502 07:58:47.032529 22501 raft_consensus.cc:2454] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:47.032568 23773 raft_consensus.cc:2726] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20240502 07:58:47.032824 22698 leader_election.cc:302] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters: b533a7d8d97943609c6e28e3e36b78a4
I20240502 07:58:47.032976 23666 raft_consensus.cc:2781] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:47.033066 23666 raft_consensus.cc:686] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:58:47.033192 23666 consensus_queue.cc:237] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.034940 22227 catalog_manager.cc:5484] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:58:47.055794 23666 raft_consensus.cc:482] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:47.055907 23666 raft_consensus.cc:504] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.056308 23666 leader_election.cc:288] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:47.056759 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b420738e4ec14fe6bc9b9adddb86ec46" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:47.056782 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b420738e4ec14fe6bc9b9adddb86ec46" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:47.056882 22370 raft_consensus.cc:2454] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:47.056951 22633 raft_consensus.cc:2454] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:47.057164 22701 leader_election.cc:302] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:47.057348 23666 raft_consensus.cc:2781] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:47.057404 23666 raft_consensus.cc:482] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:47.057442 23666 raft_consensus.cc:3037] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:47.058300 23666 raft_consensus.cc:504] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.058615 23666 leader_election.cc:288] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:47.058910 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b420738e4ec14fe6bc9b9adddb86ec46" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:47.058982 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b420738e4ec14fe6bc9b9adddb86ec46" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:47.059034 22370 raft_consensus.cc:3037] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:47.059077 22633 raft_consensus.cc:3037] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:47.059971 22370 raft_consensus.cc:2454] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:47.059973 22633 raft_consensus.cc:2454] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:47.060250 22701 leader_election.cc:302] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:47.060400 23666 raft_consensus.cc:2781] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:47.060484 23666 raft_consensus.cc:686] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:58:47.060626 23666 consensus_queue.cc:237] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.062388 22227 catalog_manager.cc:5484] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
W20240502 07:58:47.194221 23818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:47.194497 23818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:47.197799 23818 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240502 07:58:47.285758 22764 raft_consensus.cc:1261] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:47.286432 22501 raft_consensus.cc:1261] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:47.286852 23550 consensus_queue.cc:1035] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:47.287175 23550 consensus_queue.cc:1035] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:47.290351 22371 raft_consensus.cc:1261] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:47.290942 23666 consensus_queue.cc:1035] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:47.291991 22502 raft_consensus.cc:1261] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:47.292768 23431 consensus_queue.cc:1035] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:47.293963 22633 raft_consensus.cc:1261] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:47.294220 22370 raft_consensus.cc:1261] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:47.294595 23431 consensus_queue.cc:1035] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:47.294907 23431 consensus_queue.cc:1035] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:47.323469 23772 mvcc.cc:205] Tried to move back new op lower bound from 7023152034975928320 to 7023152033929334784. Current Snapshot: MvccSnapshot[applied={T|T < 7023152034975928320}]
INSERT report
rows total: 2000
time total: 116.694 ms
time per row: 0.0583469 ms
I20240502 07:58:47.355700 22230 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:47.361158 22612 tablet_service.cc:1467] Processing CreateTablet for tablet 45925e9713cb4b748efde06f77cf3702 (DEFAULT_TABLE table=rebalance_test_table_7 [id=255d8aedf8f845a1b8080105bc5b3a93]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:47.361187 22350 tablet_service.cc:1467] Processing CreateTablet for tablet 45925e9713cb4b748efde06f77cf3702 (DEFAULT_TABLE table=rebalance_test_table_7 [id=255d8aedf8f845a1b8080105bc5b3a93]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:47.361546 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45925e9713cb4b748efde06f77cf3702. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:47.361546 22350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45925e9713cb4b748efde06f77cf3702. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:47.361496 22482 tablet_service.cc:1467] Processing CreateTablet for tablet 45925e9713cb4b748efde06f77cf3702 (DEFAULT_TABLE table=rebalance_test_table_7 [id=255d8aedf8f845a1b8080105bc5b3a93]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:47.361810 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45925e9713cb4b748efde06f77cf3702. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:47.362221 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 92162f8f5fcb4baf96fa8004b7b41b9b (DEFAULT_TABLE table=rebalance_test_table_7 [id=255d8aedf8f845a1b8080105bc5b3a93]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:47.362417 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92162f8f5fcb4baf96fa8004b7b41b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:47.363834 23805 tablet_bootstrap.cc:492] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:47.364125 23803 tablet_bootstrap.cc:492] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:47.364550 23805 tablet_bootstrap.cc:654] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:47.364626 23806 tablet_bootstrap.cc:492] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:47.364957 23803 tablet_bootstrap.cc:654] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:47.365504 23806 tablet_bootstrap.cc:654] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:47.365517 23805 tablet_bootstrap.cc:492] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:47.365612 23805 ts_tablet_manager.cc:1367] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:47.365965 23803 tablet_bootstrap.cc:492] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:47.365962 23805 raft_consensus.cc:348] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.366050 23803 ts_tablet_manager.cc:1367] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:58:47.366082 23805 raft_consensus.cc:374] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:47.366124 23805 raft_consensus.cc:729] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:47.366264 23805 consensus_queue.cc:260] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.366453 23803 raft_consensus.cc:348] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.366529 23806 tablet_bootstrap.cc:492] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:47.366542 23805 ts_tablet_manager.cc:1398] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:47.366557 23803 raft_consensus.cc:374] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:47.366598 23803 raft_consensus.cc:729] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:47.366608 23806 ts_tablet_manager.cc:1367] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:47.366719 23803 consensus_queue.cc:260] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.367029 23803 ts_tablet_manager.cc:1398] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:47.367039 23806 raft_consensus.cc:348] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.367132 23806 raft_consensus.cc:374] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:47.367180 23806 raft_consensus.cc:729] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:47.367189 23803 tablet_bootstrap.cc:492] T 92162f8f5fcb4baf96fa8004b7b41b9b P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:47.367259 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 8477b961ce384edea078851122e196e9 (DEFAULT_TABLE table=rebalance_test_table_7 [id=255d8aedf8f845a1b8080105bc5b3a93]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:47.367295 23806 consensus_queue.cc:260] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.367511 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8477b961ce384edea078851122e196e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:47.367563 23806 ts_tablet_manager.cc:1398] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:47.367745 22612 tablet_service.cc:1467] Processing CreateTablet for tablet 8477b961ce384edea078851122e196e9 (DEFAULT_TABLE table=rebalance_test_table_7 [id=255d8aedf8f845a1b8080105bc5b3a93]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:47.367959 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8477b961ce384edea078851122e196e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:47.368763 23803 tablet_bootstrap.cc:654] T 92162f8f5fcb4baf96fa8004b7b41b9b P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:47.369550 22613 tablet_service.cc:1467] Processing CreateTablet for tablet 92162f8f5fcb4baf96fa8004b7b41b9b (DEFAULT_TABLE table=rebalance_test_table_7 [id=255d8aedf8f845a1b8080105bc5b3a93]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:47.369537 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 92162f8f5fcb4baf96fa8004b7b41b9b (DEFAULT_TABLE table=rebalance_test_table_7 [id=255d8aedf8f845a1b8080105bc5b3a93]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:47.369743 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92162f8f5fcb4baf96fa8004b7b41b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:47.369760 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92162f8f5fcb4baf96fa8004b7b41b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:47.370031 23803 tablet_bootstrap.cc:492] T 92162f8f5fcb4baf96fa8004b7b41b9b P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:47.370098 23803 ts_tablet_manager.cc:1367] T 92162f8f5fcb4baf96fa8004b7b41b9b P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:47.370477 23803 raft_consensus.cc:348] T 92162f8f5fcb4baf96fa8004b7b41b9b P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.370570 23803 raft_consensus.cc:374] T 92162f8f5fcb4baf96fa8004b7b41b9b P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:47.370602 23803 raft_consensus.cc:729] T 92162f8f5fcb4baf96fa8004b7b41b9b P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:47.370734 23803 consensus_queue.cc:260] T 92162f8f5fcb4baf96fa8004b7b41b9b P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.371012 23803 ts_tablet_manager.cc:1398] T 92162f8f5fcb4baf96fa8004b7b41b9b P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:47.371400 23802 tablet_bootstrap.cc:492] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:47.371582 23805 tablet_bootstrap.cc:492] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:47.372156 22351 tablet_service.cc:1467] Processing CreateTablet for tablet 8477b961ce384edea078851122e196e9 (DEFAULT_TABLE table=rebalance_test_table_7 [id=255d8aedf8f845a1b8080105bc5b3a93]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:47.372252 23802 tablet_bootstrap.cc:654] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:47.372424 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8477b961ce384edea078851122e196e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:47.372669 23805 tablet_bootstrap.cc:654] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:47.373941 23805 tablet_bootstrap.cc:492] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:47.374027 23805 ts_tablet_manager.cc:1367] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:47.374366 23805 raft_consensus.cc:348] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.374471 23805 raft_consensus.cc:374] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:47.374504 23805 raft_consensus.cc:729] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:47.374548 23806 tablet_bootstrap.cc:492] T 8477b961ce384edea078851122e196e9 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:47.374621 23805 consensus_queue.cc:260] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.374897 23805 ts_tablet_manager.cc:1398] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:47.375051 23805 tablet_bootstrap.cc:492] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:47.375439 23806 tablet_bootstrap.cc:654] T 8477b961ce384edea078851122e196e9 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:47.375946 23805 tablet_bootstrap.cc:654] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:47.376580 23802 tablet_bootstrap.cc:492] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:47.376641 23802 ts_tablet_manager.cc:1367] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:47.376996 23802 raft_consensus.cc:348] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.377106 23802 raft_consensus.cc:374] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:47.377139 23802 raft_consensus.cc:729] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:47.377346 23806 tablet_bootstrap.cc:492] T 8477b961ce384edea078851122e196e9 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:47.377277 23802 consensus_queue.cc:260] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.377426 23806 ts_tablet_manager.cc:1367] T 8477b961ce384edea078851122e196e9 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:47.377625 23802 ts_tablet_manager.cc:1398] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:47.377775 23802 tablet_bootstrap.cc:492] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:47.377890 23806 raft_consensus.cc:348] T 8477b961ce384edea078851122e196e9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.377960 23805 tablet_bootstrap.cc:492] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:47.377993 23806 raft_consensus.cc:374] T 8477b961ce384edea078851122e196e9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:47.378026 23805 ts_tablet_manager.cc:1367] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:47.378031 23806 raft_consensus.cc:729] T 8477b961ce384edea078851122e196e9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:47.378168 23806 consensus_queue.cc:260] T 8477b961ce384edea078851122e196e9 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.378461 23806 ts_tablet_manager.cc:1398] T 8477b961ce384edea078851122e196e9 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:47.378469 23805 raft_consensus.cc:348] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.378569 23805 raft_consensus.cc:374] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:47.378607 23805 raft_consensus.cc:729] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:47.378638 23802 tablet_bootstrap.cc:654] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:47.378727 23805 consensus_queue.cc:260] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.378998 23805 ts_tablet_manager.cc:1398] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:47.380300 23802 tablet_bootstrap.cc:492] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:47.380499 23802 ts_tablet_manager.cc:1367] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:47.380997 23802 raft_consensus.cc:348] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.381095 23802 raft_consensus.cc:374] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:47.381119 23802 raft_consensus.cc:729] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:47.381212 23802 consensus_queue.cc:260] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.381903 23802 ts_tablet_manager.cc:1398] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:47.443622 23726 raft_consensus.cc:482] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:47.443743 23726 raft_consensus.cc:504] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.444129 23726 leader_election.cc:288] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:47.444563 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8477b961ce384edea078851122e196e9" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:47.444449 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8477b961ce384edea078851122e196e9" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:47.444710 22764 raft_consensus.cc:2454] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:47.444878 22371 raft_consensus.cc:2454] T 8477b961ce384edea078851122e196e9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:47.445123 22570 leader_election.cc:302] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:47.445364 23726 raft_consensus.cc:2781] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:47.445459 23726 raft_consensus.cc:482] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:47.445539 23726 raft_consensus.cc:3037] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:47.446630 23726 raft_consensus.cc:504] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.446985 23726 leader_election.cc:288] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:47.447489 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8477b961ce384edea078851122e196e9" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:47.447605 22371 raft_consensus.cc:3037] T 8477b961ce384edea078851122e196e9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:47.448711 22371 raft_consensus.cc:2454] T 8477b961ce384edea078851122e196e9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:47.448923 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8477b961ce384edea078851122e196e9" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:47.448987 22570 leader_election.cc:302] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:47.449039 22764 raft_consensus.cc:3037] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:47.449546 23726 raft_consensus.cc:2781] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:47.449885 22764 raft_consensus.cc:2454] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:47.450017 23726 raft_consensus.cc:686] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:47.450170 23726 consensus_queue.cc:237] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.452095 22221 catalog_manager.cc:5484] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:58:47.481041 23854 raft_consensus.cc:482] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:47.481220 23854 raft_consensus.cc:504] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.481711 23854 leader_election.cc:288] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:47.482203 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45925e9713cb4b748efde06f77cf3702" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:47.482218 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45925e9713cb4b748efde06f77cf3702" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:47.482354 22502 raft_consensus.cc:2454] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:47.482352 22633 raft_consensus.cc:2454] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:47.482668 22305 leader_election.cc:302] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:47.482817 23854 raft_consensus.cc:2781] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:47.482877 23854 raft_consensus.cc:482] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:47.482928 23854 raft_consensus.cc:3037] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:47.483835 23854 raft_consensus.cc:504] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.484144 23854 leader_election.cc:288] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:47.484455 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45925e9713cb4b748efde06f77cf3702" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:47.484474 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45925e9713cb4b748efde06f77cf3702" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:47.484565 22633 raft_consensus.cc:3037] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:47.484571 22502 raft_consensus.cc:3037] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:47.485265 22633 raft_consensus.cc:2454] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:47.485347 22502 raft_consensus.cc:2454] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:47.485567 22308 leader_election.cc:302] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:47.485743 23854 raft_consensus.cc:2781] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:47.485823 23854 raft_consensus.cc:686] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:58:47.485985 23854 consensus_queue.cc:237] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.487869 22221 catalog_manager.cc:5484] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:58:47.600986 23851 raft_consensus.cc:482] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:47.601122 23851 raft_consensus.cc:504] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.601584 23851 leader_election.cc:288] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:47.601887 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92162f8f5fcb4baf96fa8004b7b41b9b" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:47.601986 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92162f8f5fcb4baf96fa8004b7b41b9b" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:47.602026 22633 raft_consensus.cc:2454] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:47.602126 22502 raft_consensus.cc:2454] T 92162f8f5fcb4baf96fa8004b7b41b9b P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:47.602303 22701 leader_election.cc:302] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:47.602469 23851 raft_consensus.cc:2781] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:47.602542 23851 raft_consensus.cc:482] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:47.602599 23851 raft_consensus.cc:3037] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:47.603554 23851 raft_consensus.cc:504] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.603873 23851 leader_election.cc:288] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:47.604169 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92162f8f5fcb4baf96fa8004b7b41b9b" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:47.604182 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92162f8f5fcb4baf96fa8004b7b41b9b" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:47.604290 22502 raft_consensus.cc:3037] T 92162f8f5fcb4baf96fa8004b7b41b9b P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:47.604290 22633 raft_consensus.cc:3037] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:47.605183 22633 raft_consensus.cc:2454] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:47.605180 22502 raft_consensus.cc:2454] T 92162f8f5fcb4baf96fa8004b7b41b9b P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:47.605459 22698 leader_election.cc:302] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:47.605604 23851 raft_consensus.cc:2781] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:47.605675 23851 raft_consensus.cc:686] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:58:47.605814 23851 consensus_queue.cc:237] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.607493 22221 catalog_manager.cc:5484] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
W20240502 07:58:47.770392 23873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:47.770624 23873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:47.773680 23873 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_7'
I20240502 07:58:47.858914 22502 raft_consensus.cc:1261] T 92162f8f5fcb4baf96fa8004b7b41b9b P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:47.859020 22633 raft_consensus.cc:1261] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:47.859418 23666 consensus_queue.cc:1035] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:47.859478 22502 raft_consensus.cc:1261] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:47.859586 23851 consensus_queue.cc:1035] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:47.859839 23854 consensus_queue.cc:1035] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:47.860818 22634 raft_consensus.cc:1261] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:47.861196 23854 consensus_queue.cc:1035] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:47.870492 22764 raft_consensus.cc:1261] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:47.870890 23482 consensus_queue.cc:1035] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:47.871986 22371 raft_consensus.cc:1261] T 8477b961ce384edea078851122e196e9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:47.872634 23726 consensus_queue.cc:1035] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 102.688 ms
time per row: 0.0513441 ms
I20240502 07:58:47.922583 22226 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:47.928309 22483 tablet_service.cc:1467] Processing CreateTablet for tablet c9b0511ea0124d84ac5d3996bd1efcd2 (DEFAULT_TABLE table=rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:47.928617 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9b0511ea0124d84ac5d3996bd1efcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:47.929778 22351 tablet_service.cc:1467] Processing CreateTablet for tablet c9b0511ea0124d84ac5d3996bd1efcd2 (DEFAULT_TABLE table=rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:47.930034 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9b0511ea0124d84ac5d3996bd1efcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:47.930963 22350 tablet_service.cc:1467] Processing CreateTablet for tablet 95952027a0314a2ea3f603974b96befe (DEFAULT_TABLE table=rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:47.931169 22350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95952027a0314a2ea3f603974b96befe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:47.932031 23933 tablet_bootstrap.cc:492] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:47.932125 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 12d9f49c05264a6291a7489d1675b697 (DEFAULT_TABLE table=rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:47.932312 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12d9f49c05264a6291a7489d1675b697. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:47.932906 23933 tablet_bootstrap.cc:654] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:47.932943 22612 tablet_service.cc:1467] Processing CreateTablet for tablet c9b0511ea0124d84ac5d3996bd1efcd2 (DEFAULT_TABLE table=rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:47.933205 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9b0511ea0124d84ac5d3996bd1efcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:47.933598 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 95952027a0314a2ea3f603974b96befe (DEFAULT_TABLE table=rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:47.933634 22613 tablet_service.cc:1467] Processing CreateTablet for tablet 12d9f49c05264a6291a7489d1675b697 (DEFAULT_TABLE table=rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:47.933827 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95952027a0314a2ea3f603974b96befe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:47.933828 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12d9f49c05264a6291a7489d1675b697. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:47.935204 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 12d9f49c05264a6291a7489d1675b697 (DEFAULT_TABLE table=rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:47.935410 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12d9f49c05264a6291a7489d1675b697. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:47.935830 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 95952027a0314a2ea3f603974b96befe (DEFAULT_TABLE table=rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:47.935943 23932 tablet_bootstrap.cc:492] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:47.936048 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95952027a0314a2ea3f603974b96befe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:47.936882 23932 tablet_bootstrap.cc:654] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:47.937983 23935 tablet_bootstrap.cc:492] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:47.938861 23935 tablet_bootstrap.cc:654] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:47.938966 23933 tablet_bootstrap.cc:492] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:47.939054 23933 ts_tablet_manager.cc:1367] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240502 07:58:47.939550 23933 raft_consensus.cc:348] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:47.939666 23933 raft_consensus.cc:374] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:47.939703 23933 raft_consensus.cc:729] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:47.939834 23933 consensus_queue.cc:260] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:47.940112 23933 ts_tablet_manager.cc:1398] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:47.940289 23933 tablet_bootstrap.cc:492] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:47.940366 23935 tablet_bootstrap.cc:492] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:47.940434 23935 ts_tablet_manager.cc:1367] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:47.940790 23935 raft_consensus.cc:348] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:47.940873 23935 raft_consensus.cc:374] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:47.940902 23935 raft_consensus.cc:729] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:47.941031 23935 consensus_queue.cc:260] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:47.941143 23933 tablet_bootstrap.cc:654] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:47.941771 23935 ts_tablet_manager.cc:1398] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:47.942936 23933 tablet_bootstrap.cc:492] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:47.943017 23933 ts_tablet_manager.cc:1367] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 07:58:47.943034 23938 tablet_bootstrap.cc:492] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:47.943130 23935 tablet_bootstrap.cc:492] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:47.943454 23933 raft_consensus.cc:348] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.943560 23933 raft_consensus.cc:374] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:47.943594 23933 raft_consensus.cc:729] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:47.943706 23933 consensus_queue.cc:260] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.943866 23938 tablet_bootstrap.cc:654] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:47.943995 23933 ts_tablet_manager.cc:1398] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:47.944186 23933 tablet_bootstrap.cc:492] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:47.944484 23935 tablet_bootstrap.cc:654] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:47.944943 23932 tablet_bootstrap.cc:492] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:47.945020 23932 ts_tablet_manager.cc:1367] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 07:58:47.945039 23933 tablet_bootstrap.cc:654] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:47.945461 23932 raft_consensus.cc:348] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:47.945580 23932 raft_consensus.cc:374] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:47.945609 23932 raft_consensus.cc:729] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:47.945724 23932 consensus_queue.cc:260] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:47.946041 23932 ts_tablet_manager.cc:1398] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:47.946209 23932 tablet_bootstrap.cc:492] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:47.946890 23933 tablet_bootstrap.cc:492] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:47.946970 23933 ts_tablet_manager.cc:1367] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 07:58:47.947041 23932 tablet_bootstrap.cc:654] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:47.947352 23938 tablet_bootstrap.cc:492] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:47.947427 23938 ts_tablet_manager.cc:1367] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:47.947598 23933 raft_consensus.cc:348] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.947680 23933 raft_consensus.cc:374] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:47.947710 23933 raft_consensus.cc:729] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:47.947809 23933 consensus_queue.cc:260] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.947851 23938 raft_consensus.cc:348] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.947947 23938 raft_consensus.cc:374] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:47.947978 23938 raft_consensus.cc:729] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:47.948067 23933 ts_tablet_manager.cc:1398] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:47.948401 23938 consensus_queue.cc:260] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.948741 23932 tablet_bootstrap.cc:492] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:47.948820 23932 ts_tablet_manager.cc:1367] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:47.948956 23935 tablet_bootstrap.cc:492] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:47.949021 23935 ts_tablet_manager.cc:1367] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 07:58:47.949249 23932 raft_consensus.cc:348] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.949373 23932 raft_consensus.cc:374] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:47.949407 23932 raft_consensus.cc:729] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:47.949473 23935 raft_consensus.cc:348] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.949519 23932 consensus_queue.cc:260] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.949823 23932 ts_tablet_manager.cc:1398] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:47.950419 23935 raft_consensus.cc:374] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:47.950520 23935 raft_consensus.cc:729] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:47.950662 23938 ts_tablet_manager.cc:1398] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:47.950706 23935 consensus_queue.cc:260] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:47.950902 23938 tablet_bootstrap.cc:492] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:47.951918 23935 ts_tablet_manager.cc:1398] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:47.951943 23938 tablet_bootstrap.cc:654] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:47.952965 23938 tablet_bootstrap.cc:492] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:47.953055 23938 ts_tablet_manager.cc:1367] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:47.953637 23938 raft_consensus.cc:348] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.953744 23938 raft_consensus.cc:374] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:47.953825 23938 raft_consensus.cc:729] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:47.953958 23938 consensus_queue.cc:260] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:47.954223 23938 ts_tablet_manager.cc:1398] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:48.010762 23854 raft_consensus.cc:482] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:48.010917 23854 raft_consensus.cc:504] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:48.011410 23854 leader_election.cc:288] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:48.011922 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95952027a0314a2ea3f603974b96befe" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:48.011925 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95952027a0314a2ea3f603974b96befe" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:48.012053 22502 raft_consensus.cc:2454] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:48.012084 22764 raft_consensus.cc:2454] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:48.012413 22305 leader_election.cc:302] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:48.012620 23854 raft_consensus.cc:2781] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:48.012679 23854 raft_consensus.cc:482] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:48.012728 23854 raft_consensus.cc:3037] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:48.013661 23854 raft_consensus.cc:504] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:48.014022 23854 leader_election.cc:288] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:48.014436 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95952027a0314a2ea3f603974b96befe" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:48.014429 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95952027a0314a2ea3f603974b96befe" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:48.014556 22764 raft_consensus.cc:3037] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:48.014564 22502 raft_consensus.cc:3037] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:48.015399 22764 raft_consensus.cc:2454] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:48.015408 22502 raft_consensus.cc:2454] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:48.015684 22308 leader_election.cc:302] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:48.015853 23854 raft_consensus.cc:2781] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:48.015949 23854 raft_consensus.cc:686] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:58:48.016104 23854 consensus_queue.cc:237] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:48.017863 22226 catalog_manager.cc:5484] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:58:48.242972 23666 raft_consensus.cc:482] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:48.243122 23666 raft_consensus.cc:504] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:48.243506 23666 leader_election.cc:288] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:48.243862 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12d9f49c05264a6291a7489d1675b697" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:48.244017 22371 raft_consensus.cc:2454] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:48.243988 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12d9f49c05264a6291a7489d1675b697" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:48.244143 22634 raft_consensus.cc:2454] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:48.244339 22701 leader_election.cc:302] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:48.244498 23666 raft_consensus.cc:2781] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:48.244551 23666 raft_consensus.cc:482] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:48.244577 23666 raft_consensus.cc:3037] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:48.245369 23666 raft_consensus.cc:504] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:48.245689 23666 leader_election.cc:288] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:48.246021 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12d9f49c05264a6291a7489d1675b697" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:48.246047 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12d9f49c05264a6291a7489d1675b697" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:48.246124 22371 raft_consensus.cc:3037] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:48.246136 22634 raft_consensus.cc:3037] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:48.246831 22371 raft_consensus.cc:2454] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:48.246846 22634 raft_consensus.cc:2454] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:48.247089 22701 leader_election.cc:302] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:48.247314 23666 raft_consensus.cc:2781] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:48.247419 23666 raft_consensus.cc:686] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:58:48.247567 23666 consensus_queue.cc:237] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:48.249215 22226 catalog_manager.cc:5484] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:58:48.314126 23482 raft_consensus.cc:482] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:48.314285 23482 raft_consensus.cc:504] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:48.314738 23482 leader_election.cc:288] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:48.315192 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9b0511ea0124d84ac5d3996bd1efcd2" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:48.315191 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9b0511ea0124d84ac5d3996bd1efcd2" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:48.315321 22502 raft_consensus.cc:2454] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:48.315333 22370 raft_consensus.cc:2454] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:48.315634 22570 leader_election.cc:302] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:48.315797 23482 raft_consensus.cc:2781] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:48.315873 23482 raft_consensus.cc:482] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:48.315913 23482 raft_consensus.cc:3037] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:48.316702 23482 raft_consensus.cc:504] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:48.317019 23482 leader_election.cc:288] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:48.317354 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9b0511ea0124d84ac5d3996bd1efcd2" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:48.317409 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9b0511ea0124d84ac5d3996bd1efcd2" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:48.317472 22502 raft_consensus.cc:3037] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:48.317531 22370 raft_consensus.cc:3037] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:48.318279 22502 raft_consensus.cc:2454] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:48.318296 22370 raft_consensus.cc:2454] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:48.318564 22567 leader_election.cc:302] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:48.318735 23482 raft_consensus.cc:2781] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:48.318832 23482 raft_consensus.cc:686] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:48.318976 23482 consensus_queue.cc:237] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:48.320677 22226 catalog_manager.cc:5484] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:58:48.406494 23854 consensus_queue.cc:1035] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:48.406706 23853 consensus_queue.cc:1035] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 07:58:48.500855 23948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:48.501089 23948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:48.504055 23948 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_8'
I20240502 07:58:48.607563 22370 raft_consensus.cc:1261] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:48.607606 22634 raft_consensus.cc:1261] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:48.611138 23666 consensus_queue.cc:1035] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:48.611320 23851 consensus_queue.cc:1035] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:48.622078 22370 raft_consensus.cc:1261] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:48.622602 23482 consensus_queue.cc:1035] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:48.625353 22502 raft_consensus.cc:1261] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:48.626001 23482 consensus_queue.cc:1035] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 131.938 ms
time per row: 0.0659689 ms
I20240502 07:58:48.677279 22230 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:48.683017 22745 tablet_service.cc:1467] Processing CreateTablet for tablet a0f577dafc0b4846a1a9e4dc54a5a687 (DEFAULT_TABLE table=rebalance_test_table_9 [id=9b69b771dcfb4a00a937ed9203245903]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:48.683081 22483 tablet_service.cc:1467] Processing CreateTablet for tablet a0f577dafc0b4846a1a9e4dc54a5a687 (DEFAULT_TABLE table=rebalance_test_table_9 [id=9b69b771dcfb4a00a937ed9203245903]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:48.683353 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0f577dafc0b4846a1a9e4dc54a5a687. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:48.683354 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0f577dafc0b4846a1a9e4dc54a5a687. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:48.683584 22482 tablet_service.cc:1467] Processing CreateTablet for tablet a0430566f5ae4ffabcf15bb8b8af6bb5 (DEFAULT_TABLE table=rebalance_test_table_9 [id=9b69b771dcfb4a00a937ed9203245903]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:48.684135 22612 tablet_service.cc:1467] Processing CreateTablet for tablet a0430566f5ae4ffabcf15bb8b8af6bb5 (DEFAULT_TABLE table=rebalance_test_table_9 [id=9b69b771dcfb4a00a937ed9203245903]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:48.684226 22352 tablet_service.cc:1467] Processing CreateTablet for tablet b7cdcee32c8f4cd3af30c402b8da2484 (DEFAULT_TABLE table=rebalance_test_table_9 [id=9b69b771dcfb4a00a937ed9203245903]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:48.684398 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0430566f5ae4ffabcf15bb8b8af6bb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:48.684502 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7cdcee32c8f4cd3af30c402b8da2484. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:48.684978 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0430566f5ae4ffabcf15bb8b8af6bb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:48.686583 22614 tablet_service.cc:1467] Processing CreateTablet for tablet b7cdcee32c8f4cd3af30c402b8da2484 (DEFAULT_TABLE table=rebalance_test_table_9 [id=9b69b771dcfb4a00a937ed9203245903]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:48.686775 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7cdcee32c8f4cd3af30c402b8da2484. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:48.686934 24013 tablet_bootstrap.cc:492] T b7cdcee32c8f4cd3af30c402b8da2484 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:48.687794 24013 tablet_bootstrap.cc:654] T b7cdcee32c8f4cd3af30c402b8da2484 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:48.688742 22743 tablet_service.cc:1467] Processing CreateTablet for tablet a0430566f5ae4ffabcf15bb8b8af6bb5 (DEFAULT_TABLE table=rebalance_test_table_9 [id=9b69b771dcfb4a00a937ed9203245903]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:48.684129 22613 tablet_service.cc:1467] Processing CreateTablet for tablet a0f577dafc0b4846a1a9e4dc54a5a687 (DEFAULT_TABLE table=rebalance_test_table_9 [id=9b69b771dcfb4a00a937ed9203245903]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:48.688980 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0430566f5ae4ffabcf15bb8b8af6bb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:48.688979 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0f577dafc0b4846a1a9e4dc54a5a687. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:48.689442 24012 tablet_bootstrap.cc:492] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:48.689600 22482 tablet_service.cc:1467] Processing CreateTablet for tablet b7cdcee32c8f4cd3af30c402b8da2484 (DEFAULT_TABLE table=rebalance_test_table_9 [id=9b69b771dcfb4a00a937ed9203245903]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:48.689822 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7cdcee32c8f4cd3af30c402b8da2484. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:48.690402 24012 tablet_bootstrap.cc:654] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:48.691149 24015 tablet_bootstrap.cc:492] T a0f577dafc0b4846a1a9e4dc54a5a687 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:48.691818 24011 tablet_bootstrap.cc:492] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:48.692071 24015 tablet_bootstrap.cc:654] T a0f577dafc0b4846a1a9e4dc54a5a687 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:48.692734 24011 tablet_bootstrap.cc:654] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:48.692961 24013 tablet_bootstrap.cc:492] T b7cdcee32c8f4cd3af30c402b8da2484 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:48.693055 24013 ts_tablet_manager.cc:1367] T b7cdcee32c8f4cd3af30c402b8da2484 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:58:48.693554 24013 raft_consensus.cc:348] T b7cdcee32c8f4cd3af30c402b8da2484 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:48.693765 24013 raft_consensus.cc:374] T b7cdcee32c8f4cd3af30c402b8da2484 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:48.693838 24013 raft_consensus.cc:729] T b7cdcee32c8f4cd3af30c402b8da2484 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:48.693905 24011 tablet_bootstrap.cc:492] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:48.693989 24011 ts_tablet_manager.cc:1367] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:48.694456 24011 raft_consensus.cc:348] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:48.694551 24011 raft_consensus.cc:374] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:48.694588 24011 raft_consensus.cc:729] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:48.694566 24013 consensus_queue.cc:260] T b7cdcee32c8f4cd3af30c402b8da2484 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:48.694693 24011 consensus_queue.cc:260] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:48.694934 24011 ts_tablet_manager.cc:1398] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:48.694933 24013 ts_tablet_manager.cc:1398] T b7cdcee32c8f4cd3af30c402b8da2484 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:48.695103 24011 tablet_bootstrap.cc:492] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:48.696153 24011 tablet_bootstrap.cc:654] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:48.698160 24011 tablet_bootstrap.cc:492] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:48.698272 24011 ts_tablet_manager.cc:1367] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:48.698288 24012 tablet_bootstrap.cc:492] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:48.698362 24012 ts_tablet_manager.cc:1367] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 07:58:48.698710 24011 raft_consensus.cc:348] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:48.698837 24011 raft_consensus.cc:374] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:48.698786 24012 raft_consensus.cc:348] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:48.698889 24011 raft_consensus.cc:729] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:48.698923 24012 raft_consensus.cc:374] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:48.698980 24012 raft_consensus.cc:729] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:48.699062 24011 consensus_queue.cc:260] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:48.699126 24012 consensus_queue.cc:260] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:48.699429 24011 ts_tablet_manager.cc:1398] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:48.699599 24011 tablet_bootstrap.cc:492] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:48.699918 24012 ts_tablet_manager.cc:1398] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:48.700099 24012 tablet_bootstrap.cc:492] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:48.700685 24011 tablet_bootstrap.cc:654] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:48.701013 24012 tablet_bootstrap.cc:654] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:48.701857 24015 tablet_bootstrap.cc:492] T a0f577dafc0b4846a1a9e4dc54a5a687 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:48.701956 24015 ts_tablet_manager.cc:1367] T a0f577dafc0b4846a1a9e4dc54a5a687 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 07:58:48.702373 24015 raft_consensus.cc:348] T a0f577dafc0b4846a1a9e4dc54a5a687 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:48.702471 24015 raft_consensus.cc:374] T a0f577dafc0b4846a1a9e4dc54a5a687 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:48.702499 24015 raft_consensus.cc:729] T a0f577dafc0b4846a1a9e4dc54a5a687 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:48.702620 24015 consensus_queue.cc:260] T a0f577dafc0b4846a1a9e4dc54a5a687 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:48.702900 24015 ts_tablet_manager.cc:1398] T a0f577dafc0b4846a1a9e4dc54a5a687 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:48.702991 24011 tablet_bootstrap.cc:492] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:48.703085 24011 ts_tablet_manager.cc:1367] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:48.703583 24011 raft_consensus.cc:348] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:48.703860 24011 raft_consensus.cc:374] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:48.703902 24011 raft_consensus.cc:729] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:48.703902 24015 tablet_bootstrap.cc:492] T a0430566f5ae4ffabcf15bb8b8af6bb5 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:48.704057 24011 consensus_queue.cc:260] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:48.704378 24011 ts_tablet_manager.cc:1398] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:48.704785 24015 tablet_bootstrap.cc:654] T a0430566f5ae4ffabcf15bb8b8af6bb5 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:48.705212 24012 tablet_bootstrap.cc:492] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:48.705292 24012 ts_tablet_manager.cc:1367] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:48.706151 24012 raft_consensus.cc:348] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:48.706308 24012 raft_consensus.cc:374] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:48.706383 24012 raft_consensus.cc:729] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:48.706452 24015 tablet_bootstrap.cc:492] T a0430566f5ae4ffabcf15bb8b8af6bb5 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:48.706565 24015 ts_tablet_manager.cc:1367] T a0430566f5ae4ffabcf15bb8b8af6bb5 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:48.706578 24012 consensus_queue.cc:260] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:48.707023 24015 raft_consensus.cc:348] T a0430566f5ae4ffabcf15bb8b8af6bb5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:48.707201 24015 raft_consensus.cc:374] T a0430566f5ae4ffabcf15bb8b8af6bb5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:48.707285 24015 raft_consensus.cc:729] T a0430566f5ae4ffabcf15bb8b8af6bb5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:48.707445 24012 ts_tablet_manager.cc:1398] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:48.707446 24015 consensus_queue.cc:260] T a0430566f5ae4ffabcf15bb8b8af6bb5 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:48.707628 24012 tablet_bootstrap.cc:492] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:48.707777 24015 ts_tablet_manager.cc:1398] T a0430566f5ae4ffabcf15bb8b8af6bb5 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:48.708472 24012 tablet_bootstrap.cc:654] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:48.709551 24012 tablet_bootstrap.cc:492] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:48.709687 24012 ts_tablet_manager.cc:1367] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:48.710151 24012 raft_consensus.cc:348] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:48.710244 24012 raft_consensus.cc:374] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:48.710281 24012 raft_consensus.cc:729] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:48.710427 24012 consensus_queue.cc:260] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:48.710718 24012 ts_tablet_manager.cc:1398] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:48.730111 24006 raft_consensus.cc:482] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:48.730243 24006 raft_consensus.cc:504] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:48.730686 24006 leader_election.cc:288] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:48.731103 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7cdcee32c8f4cd3af30c402b8da2484" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:48.731199 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7cdcee32c8f4cd3af30c402b8da2484" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:48.731251 22370 raft_consensus.cc:2454] T b7cdcee32c8f4cd3af30c402b8da2484 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:48.731334 22502 raft_consensus.cc:2454] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:48.731580 22570 leader_election.cc:302] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:48.731753 24006 raft_consensus.cc:2781] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:48.731828 24006 raft_consensus.cc:482] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:48.731868 24006 raft_consensus.cc:3037] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:48.733000 24006 raft_consensus.cc:504] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:48.733445 24006 leader_election.cc:288] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:48.733731 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7cdcee32c8f4cd3af30c402b8da2484" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:48.733716 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7cdcee32c8f4cd3af30c402b8da2484" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:48.733834 22502 raft_consensus.cc:3037] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:48.733841 22370 raft_consensus.cc:3037] T b7cdcee32c8f4cd3af30c402b8da2484 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:48.734575 22502 raft_consensus.cc:2454] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:48.734747 22370 raft_consensus.cc:2454] T b7cdcee32c8f4cd3af30c402b8da2484 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:48.734848 22567 leader_election.cc:302] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:48.734997 24006 raft_consensus.cc:2781] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:48.735076 24006 raft_consensus.cc:686] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:48.735220 24006 consensus_queue.cc:237] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:48.737214 22221 catalog_manager.cc:5484] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:58:48.860438 23792 raft_consensus.cc:482] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:48.860569 23792 raft_consensus.cc:504] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:48.860950 23792 leader_election.cc:288] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:48.861399 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0430566f5ae4ffabcf15bb8b8af6bb5" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:48.861430 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0430566f5ae4ffabcf15bb8b8af6bb5" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:48.861541 22634 raft_consensus.cc:2454] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:48.861603 22764 raft_consensus.cc:2454] T a0430566f5ae4ffabcf15bb8b8af6bb5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:48.861887 22439 leader_election.cc:302] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:48.862066 23792 raft_consensus.cc:2781] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:48.862167 23792 raft_consensus.cc:482] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:48.862223 23792 raft_consensus.cc:3037] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:48.863188 23792 raft_consensus.cc:504] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:48.863505 23792 leader_election.cc:288] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:48.863852 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0430566f5ae4ffabcf15bb8b8af6bb5" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:48.863853 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0430566f5ae4ffabcf15bb8b8af6bb5" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:48.863965 22634 raft_consensus.cc:3037] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:48.863970 22764 raft_consensus.cc:3037] T a0430566f5ae4ffabcf15bb8b8af6bb5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:48.864710 22764 raft_consensus.cc:2454] T a0430566f5ae4ffabcf15bb8b8af6bb5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:58:48.864794 22634 raft_consensus.cc:2454] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:58:48.864957 22439 leader_election.cc:302] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:48.865132 23792 raft_consensus.cc:2781] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:48.865366 23792 raft_consensus.cc:686] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:58:48.865515 23792 consensus_queue.cc:237] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:48.867208 22221 catalog_manager.cc:5484] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:58:49.013895 24006 raft_consensus.cc:482] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:49.014046 24006 raft_consensus.cc:504] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:49.014448 24006 leader_election.cc:288] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:49.014832 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0f577dafc0b4846a1a9e4dc54a5a687" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:49.014871 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0f577dafc0b4846a1a9e4dc54a5a687" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:49.014988 22764 raft_consensus.cc:2454] T a0f577dafc0b4846a1a9e4dc54a5a687 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:49.014986 22502 raft_consensus.cc:2454] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:49.015300 22567 leader_election.cc:302] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:49.015480 24006 raft_consensus.cc:2781] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:49.015539 24006 raft_consensus.cc:482] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:49.015573 24006 raft_consensus.cc:3037] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:49.016399 24006 raft_consensus.cc:504] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:49.016708 24006 leader_election.cc:288] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:49.017052 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0f577dafc0b4846a1a9e4dc54a5a687" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:49.017063 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0f577dafc0b4846a1a9e4dc54a5a687" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:49.017150 22502 raft_consensus.cc:3037] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:49.017182 22764 raft_consensus.cc:3037] T a0f577dafc0b4846a1a9e4dc54a5a687 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:49.017915 22502 raft_consensus.cc:2454] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:49.017938 22764 raft_consensus.cc:2454] T a0f577dafc0b4846a1a9e4dc54a5a687 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:49.018177 22567 leader_election.cc:302] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:49.018429 24006 raft_consensus.cc:2781] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:49.018507 24006 raft_consensus.cc:686] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:49.018903 24006 consensus_queue.cc:237] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:49.021668 22221 catalog_manager.cc:5484] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:58:49.132694 24006 consensus_queue.cc:1035] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:49.136111 24006 consensus_queue.cc:1035] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 07:58:49.174656 24028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:49.174911 24028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:49.178068 24028 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_9'
I20240502 07:58:49.261817 22764 raft_consensus.cc:1261] T a0430566f5ae4ffabcf15bb8b8af6bb5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:49.261817 22634 raft_consensus.cc:1261] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:49.262295 24025 consensus_queue.cc:1035] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:49.263036 22501 raft_consensus.cc:1261] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:49.264181 22764 raft_consensus.cc:1261] T a0f577dafc0b4846a1a9e4dc54a5a687 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:49.270236 23792 consensus_queue.cc:1035] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:49.269243 24065 consensus_queue.cc:1035] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:49.273440 24065 consensus_queue.cc:1035] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 106.07 ms
time per row: 0.0530348 ms
I20240502 07:58:49.333175 22226 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:49.339381 22743 tablet_service.cc:1467] Processing CreateTablet for tablet d0e0d340b11e4570b298e9ec75bd4e54 (DEFAULT_TABLE table=rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:49.339728 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0e0d340b11e4570b298e9ec75bd4e54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:49.339694 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 6d956ab18c8e479a9211afa3563dbe6f (DEFAULT_TABLE table=rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:49.339941 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d956ab18c8e479a9211afa3563dbe6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:49.340086 22352 tablet_service.cc:1467] Processing CreateTablet for tablet d0e0d340b11e4570b298e9ec75bd4e54 (DEFAULT_TABLE table=rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:49.340258 22482 tablet_service.cc:1467] Processing CreateTablet for tablet d0e0d340b11e4570b298e9ec75bd4e54 (DEFAULT_TABLE table=rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:49.340368 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0e0d340b11e4570b298e9ec75bd4e54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:49.340366 22351 tablet_service.cc:1467] Processing CreateTablet for tablet d4c1fadf79ab475b83e3303918f96d46 (DEFAULT_TABLE table=rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:49.340523 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0e0d340b11e4570b298e9ec75bd4e54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:49.340606 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4c1fadf79ab475b83e3303918f96d46. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:49.341593 22350 tablet_service.cc:1467] Processing CreateTablet for tablet 6d956ab18c8e479a9211afa3563dbe6f (DEFAULT_TABLE table=rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:49.341706 22613 tablet_service.cc:1467] Processing CreateTablet for tablet 6d956ab18c8e479a9211afa3563dbe6f (DEFAULT_TABLE table=rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:49.341823 22350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d956ab18c8e479a9211afa3563dbe6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:49.341956 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d956ab18c8e479a9211afa3563dbe6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:49.342865 24088 tablet_bootstrap.cc:492] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:49.343544 24088 tablet_bootstrap.cc:654] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:49.344204 22483 tablet_service.cc:1467] Processing CreateTablet for tablet d4c1fadf79ab475b83e3303918f96d46 (DEFAULT_TABLE table=rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:49.344377 22744 tablet_service.cc:1467] Processing CreateTablet for tablet d4c1fadf79ab475b83e3303918f96d46 (DEFAULT_TABLE table=rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:49.344458 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4c1fadf79ab475b83e3303918f96d46. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:49.344580 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4c1fadf79ab475b83e3303918f96d46. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:49.344719 24088 tablet_bootstrap.cc:492] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:49.344805 24088 ts_tablet_manager.cc:1367] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:49.345273 24088 raft_consensus.cc:348] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:49.345427 24088 raft_consensus.cc:374] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:49.345463 24088 raft_consensus.cc:729] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:49.345592 24088 consensus_queue.cc:260] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:49.345963 24088 ts_tablet_manager.cc:1398] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:49.346130 24088 tablet_bootstrap.cc:492] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:49.347065 24088 tablet_bootstrap.cc:654] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:49.347744 24091 tablet_bootstrap.cc:492] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:49.348047 24088 tablet_bootstrap.cc:492] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:49.348031 24087 tablet_bootstrap.cc:492] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:49.348130 24088 ts_tablet_manager.cc:1367] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:49.348531 24088 raft_consensus.cc:348] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.348624 24088 raft_consensus.cc:374] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:49.348654 24088 raft_consensus.cc:729] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:49.348758 24088 consensus_queue.cc:260] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.348939 24088 ts_tablet_manager.cc:1398] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:49.349018 24091 tablet_bootstrap.cc:654] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:49.349074 24088 tablet_bootstrap.cc:492] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:49.350041 24091 tablet_bootstrap.cc:492] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:49.350127 24091 ts_tablet_manager.cc:1367] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:49.350509 24087 tablet_bootstrap.cc:654] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:49.350587 24091 raft_consensus.cc:348] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.350675 24091 raft_consensus.cc:374] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:49.350703 24091 raft_consensus.cc:729] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:49.350834 24088 tablet_bootstrap.cc:654] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:49.350816 24091 consensus_queue.cc:260] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.351097 24091 ts_tablet_manager.cc:1398] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:49.351258 24091 tablet_bootstrap.cc:492] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:49.351583 24087 tablet_bootstrap.cc:492] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:49.351658 24087 ts_tablet_manager.cc:1367] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:49.352094 24091 tablet_bootstrap.cc:654] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:49.352072 24087 raft_consensus.cc:348] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.352176 24087 raft_consensus.cc:374] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:49.352218 24087 raft_consensus.cc:729] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:49.352340 24087 consensus_queue.cc:260] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.352588 24088 tablet_bootstrap.cc:492] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:49.352612 24087 ts_tablet_manager.cc:1398] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:49.352656 24088 ts_tablet_manager.cc:1367] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:49.352800 24087 tablet_bootstrap.cc:492] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:49.353065 24088 raft_consensus.cc:348] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.353178 24088 raft_consensus.cc:374] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:49.353214 24088 raft_consensus.cc:729] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:49.353500 24091 tablet_bootstrap.cc:492] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:49.353572 24091 ts_tablet_manager.cc:1367] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:49.353678 24087 tablet_bootstrap.cc:654] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:49.353610 24088 consensus_queue.cc:260] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.353906 24088 ts_tablet_manager.cc:1398] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:49.354010 24091 raft_consensus.cc:348] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.354102 24091 raft_consensus.cc:374] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:49.354135 24091 raft_consensus.cc:729] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:49.354241 24091 consensus_queue.cc:260] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.354523 24091 ts_tablet_manager.cc:1398] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:49.355293 24090 tablet_bootstrap.cc:492] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:49.355755 24087 tablet_bootstrap.cc:492] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:49.355901 24087 ts_tablet_manager.cc:1367] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:49.356091 24090 tablet_bootstrap.cc:654] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:49.356357 24087 raft_consensus.cc:348] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:49.356485 24087 raft_consensus.cc:374] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:49.356535 24087 raft_consensus.cc:729] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:49.356691 24087 consensus_queue.cc:260] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:49.357492 24090 tablet_bootstrap.cc:492] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:49.357589 24090 ts_tablet_manager.cc:1367] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:49.357682 24087 ts_tablet_manager.cc:1398] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:49.357903 24087 tablet_bootstrap.cc:492] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:49.358040 24090 raft_consensus.cc:348] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:49.358161 24090 raft_consensus.cc:374] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:49.358204 24090 raft_consensus.cc:729] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:49.358337 24090 consensus_queue.cc:260] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:49.358628 24090 ts_tablet_manager.cc:1398] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:49.359194 24087 tablet_bootstrap.cc:654] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:49.360601 24087 tablet_bootstrap.cc:492] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:49.360682 24087 ts_tablet_manager.cc:1367] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:49.361109 24087 raft_consensus.cc:348] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.361210 24087 raft_consensus.cc:374] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:49.361232 24087 raft_consensus.cc:729] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:49.361361 24087 consensus_queue.cc:260] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.361616 24087 ts_tablet_manager.cc:1398] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:49.413342 23792 raft_consensus.cc:482] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:49.413491 23792 raft_consensus.cc:504] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.413892 23792 leader_election.cc:288] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:49.414391 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0e0d340b11e4570b298e9ec75bd4e54" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:49.414386 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0e0d340b11e4570b298e9ec75bd4e54" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:49.414536 22765 raft_consensus.cc:2454] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:49.414536 22370 raft_consensus.cc:2454] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:49.414858 22439 leader_election.cc:302] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:49.415025 23792 raft_consensus.cc:2781] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:49.415097 23792 raft_consensus.cc:482] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:49.415146 23792 raft_consensus.cc:3037] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:49.416036 23792 raft_consensus.cc:504] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.416430 23792 leader_election.cc:288] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:49.416877 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0e0d340b11e4570b298e9ec75bd4e54" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:49.416900 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0e0d340b11e4570b298e9ec75bd4e54" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:49.417019 22370 raft_consensus.cc:3037] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:49.417028 22765 raft_consensus.cc:3037] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:49.417861 22370 raft_consensus.cc:2454] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:58:49.418038 22765 raft_consensus.cc:2454] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:58:49.418156 22439 leader_election.cc:302] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:49.418329 23792 raft_consensus.cc:2781] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:49.418437 23792 raft_consensus.cc:686] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:58:49.418573 23792 consensus_queue.cc:237] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.420433 22226 catalog_manager.cc:5484] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:58:49.446796 23792 raft_consensus.cc:482] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:49.446923 23792 raft_consensus.cc:504] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.447263 23792 leader_election.cc:288] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:49.447691 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4c1fadf79ab475b83e3303918f96d46" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:49.447718 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4c1fadf79ab475b83e3303918f96d46" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:49.447834 22370 raft_consensus.cc:2454] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:49.447832 22765 raft_consensus.cc:2454] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:49.448134 22439 leader_election.cc:302] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:49.448316 23792 raft_consensus.cc:2781] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:49.448369 23792 raft_consensus.cc:482] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:49.448397 23792 raft_consensus.cc:3037] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:49.449213 23792 raft_consensus.cc:504] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.449568 23792 leader_election.cc:288] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:49.449885 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4c1fadf79ab475b83e3303918f96d46" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:49.449908 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4c1fadf79ab475b83e3303918f96d46" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:49.450001 22370 raft_consensus.cc:3037] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:49.450001 22765 raft_consensus.cc:3037] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:49.450995 22370 raft_consensus.cc:2454] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:58:49.451066 22765 raft_consensus.cc:2454] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:58:49.451315 22439 leader_election.cc:302] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:49.451534 23792 raft_consensus.cc:2781] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:49.451709 23792 raft_consensus.cc:686] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:58:49.452297 23792 consensus_queue.cc:237] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.454625 22226 catalog_manager.cc:5484] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:58:49.536830 23851 raft_consensus.cc:482] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:49.536939 23851 raft_consensus.cc:504] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:49.537312 23851 leader_election.cc:288] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:49.537734 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d956ab18c8e479a9211afa3563dbe6f" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:49.537837 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d956ab18c8e479a9211afa3563dbe6f" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:49.537896 22370 raft_consensus.cc:2454] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:49.538022 22634 raft_consensus.cc:2454] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:49.538194 22701 leader_election.cc:302] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:49.538355 23851 raft_consensus.cc:2781] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:49.538419 23851 raft_consensus.cc:482] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:49.538475 23851 raft_consensus.cc:3037] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:49.539290 23851 raft_consensus.cc:504] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:49.539589 23851 leader_election.cc:288] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:49.539943 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d956ab18c8e479a9211afa3563dbe6f" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:49.540004 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d956ab18c8e479a9211afa3563dbe6f" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:49.540045 22370 raft_consensus.cc:3037] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:49.540117 22634 raft_consensus.cc:3037] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:49.540760 22370 raft_consensus.cc:2454] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:49.540823 22634 raft_consensus.cc:2454] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:49.541069 22701 leader_election.cc:302] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:49.541237 23851 raft_consensus.cc:2781] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:49.541357 23851 raft_consensus.cc:686] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:58:49.541510 23851 consensus_queue.cc:237] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:49.543254 22226 catalog_manager.cc:5484] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
W20240502 07:58:49.689141 24103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:49.689419 24103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:49.692495 24103 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_10'
I20240502 07:58:49.775560 22371 raft_consensus.cc:1261] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:49.776139 22765 raft_consensus.cc:1261] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:49.776644 24025 consensus_queue.cc:1035] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:49.776966 24025 consensus_queue.cc:1035] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:49.778347 22370 raft_consensus.cc:1261] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:49.778904 23851 consensus_queue.cc:1035] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:49.780176 22634 raft_consensus.cc:1261] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:49.781025 23851 consensus_queue.cc:1035] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:49.782039 22765 raft_consensus.cc:1261] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:49.782593 22372 raft_consensus.cc:1261] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:49.783011 23792 consensus_queue.cc:1035] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:49.783296 23792 consensus_queue.cc:1035] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:49.789582 23953 mvcc.cc:205] Tried to move back new op lower bound from 7023152045155033088 to 7023152043838218240. Current Snapshot: MvccSnapshot[applied={T|T < 7023152045155033088}]
I20240502 07:58:49.813886 24062 mvcc.cc:205] Tried to move back new op lower bound from 7023152045166313472 to 7023152044203470848. Current Snapshot: MvccSnapshot[applied={T|T < 7023152045166313472}]
I20240502 07:58:49.816519 24071 mvcc.cc:205] Tried to move back new op lower bound from 7023152045165080576 to 7023152043699765248. Current Snapshot: MvccSnapshot[applied={T|T < 7023152045165080576}]
INSERT report
rows total: 2000
time total: 116.735 ms
time per row: 0.0583675 ms
I20240502 07:58:49.851756 22226 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:49.857098 22351 tablet_service.cc:1467] Processing CreateTablet for tablet af902251d1624f95893f9bb29795d126 (DEFAULT_TABLE table=rebalance_test_table_11 [id=423ebbc6961345159cc3d4c9668fe1dd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:49.857229 22482 tablet_service.cc:1467] Processing CreateTablet for tablet af902251d1624f95893f9bb29795d126 (DEFAULT_TABLE table=rebalance_test_table_11 [id=423ebbc6961345159cc3d4c9668fe1dd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:49.857460 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af902251d1624f95893f9bb29795d126. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:49.857539 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af902251d1624f95893f9bb29795d126. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:49.858283 22613 tablet_service.cc:1467] Processing CreateTablet for tablet af902251d1624f95893f9bb29795d126 (DEFAULT_TABLE table=rebalance_test_table_11 [id=423ebbc6961345159cc3d4c9668fe1dd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:49.858327 22614 tablet_service.cc:1467] Processing CreateTablet for tablet 423d6be23b044130b0c916f995dffced (DEFAULT_TABLE table=rebalance_test_table_11 [id=423ebbc6961345159cc3d4c9668fe1dd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:49.858556 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af902251d1624f95893f9bb29795d126. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:49.858628 22612 tablet_service.cc:1467] Processing CreateTablet for tablet 2241a741bf85433b9fea512e9c530416 (DEFAULT_TABLE table=rebalance_test_table_11 [id=423ebbc6961345159cc3d4c9668fe1dd]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:49.859230 22744 tablet_service.cc:1467] Processing CreateTablet for tablet 423d6be23b044130b0c916f995dffced (DEFAULT_TABLE table=rebalance_test_table_11 [id=423ebbc6961345159cc3d4c9668fe1dd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:49.859503 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 423d6be23b044130b0c916f995dffced. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:49.859532 24087 tablet_bootstrap.cc:492] T af902251d1624f95893f9bb29795d126 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:49.860343 22482 tablet_service.cc:1467] Processing CreateTablet for tablet 2241a741bf85433b9fea512e9c530416 (DEFAULT_TABLE table=rebalance_test_table_11 [id=423ebbc6961345159cc3d4c9668fe1dd]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:49.860421 24087 tablet_bootstrap.cc:654] T af902251d1624f95893f9bb29795d126 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:49.860569 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2241a741bf85433b9fea512e9c530416. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:49.861635 24147 tablet_bootstrap.cc:492] T 423d6be23b044130b0c916f995dffced P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:49.861824 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 2241a741bf85433b9fea512e9c530416 (DEFAULT_TABLE table=rebalance_test_table_11 [id=423ebbc6961345159cc3d4c9668fe1dd]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:49.862066 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2241a741bf85433b9fea512e9c530416. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:49.862174 24087 tablet_bootstrap.cc:492] T af902251d1624f95893f9bb29795d126 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:49.862246 24087 ts_tablet_manager.cc:1367] T af902251d1624f95893f9bb29795d126 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:49.862680 24087 raft_consensus.cc:348] T af902251d1624f95893f9bb29795d126 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:49.862782 24087 raft_consensus.cc:374] T af902251d1624f95893f9bb29795d126 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:49.862813 24087 raft_consensus.cc:729] T af902251d1624f95893f9bb29795d126 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:49.862941 24087 consensus_queue.cc:260] T af902251d1624f95893f9bb29795d126 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:49.863216 24087 ts_tablet_manager.cc:1398] T af902251d1624f95893f9bb29795d126 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:49.863368 24147 tablet_bootstrap.cc:654] T 423d6be23b044130b0c916f995dffced P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:49.864235 24147 tablet_bootstrap.cc:492] T 423d6be23b044130b0c916f995dffced P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:49.864305 24147 ts_tablet_manager.cc:1367] T 423d6be23b044130b0c916f995dffced P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:49.864745 22351 tablet_service.cc:1467] Processing CreateTablet for tablet 423d6be23b044130b0c916f995dffced (DEFAULT_TABLE table=rebalance_test_table_11 [id=423ebbc6961345159cc3d4c9668fe1dd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:49.864730 24147 raft_consensus.cc:348] T 423d6be23b044130b0c916f995dffced P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.864848 24147 raft_consensus.cc:374] T 423d6be23b044130b0c916f995dffced P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:49.864879 24147 raft_consensus.cc:729] T 423d6be23b044130b0c916f995dffced P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:49.864980 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 423d6be23b044130b0c916f995dffced. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:49.864972 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2241a741bf85433b9fea512e9c530416. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:49.865029 24147 consensus_queue.cc:260] T 423d6be23b044130b0c916f995dffced P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.866778 24147 ts_tablet_manager.cc:1398] T 423d6be23b044130b0c916f995dffced P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:49.866995 24147 tablet_bootstrap.cc:492] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:49.867204 24087 tablet_bootstrap.cc:492] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:49.867506 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 423d6be23b044130b0c916f995dffced. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:49.868047 24087 tablet_bootstrap.cc:654] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:49.868651 24147 tablet_bootstrap.cc:654] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:49.869158 24087 tablet_bootstrap.cc:492] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:49.869233 24087 ts_tablet_manager.cc:1367] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:49.869714 24087 raft_consensus.cc:348] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.869802 24147 tablet_bootstrap.cc:492] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:49.869828 24087 raft_consensus.cc:374] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:49.869860 24087 raft_consensus.cc:729] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:49.869871 24147 ts_tablet_manager.cc:1367] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:49.869983 24087 consensus_queue.cc:260] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.870030 24149 tablet_bootstrap.cc:492] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:49.870245 24087 ts_tablet_manager.cc:1398] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:49.870263 24147 raft_consensus.cc:348] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:49.870353 24147 raft_consensus.cc:374] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:49.870391 24147 raft_consensus.cc:729] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:49.870502 24147 consensus_queue.cc:260] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:49.870781 24147 ts_tablet_manager.cc:1398] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:49.871318 24149 tablet_bootstrap.cc:654] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:49.871951 24145 tablet_bootstrap.cc:492] T af902251d1624f95893f9bb29795d126 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:49.872825 24145 tablet_bootstrap.cc:654] T af902251d1624f95893f9bb29795d126 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:49.873747 24149 tablet_bootstrap.cc:492] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:49.873850 24149 ts_tablet_manager.cc:1367] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:49.874264 24149 raft_consensus.cc:348] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:49.874348 24149 raft_consensus.cc:374] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:49.874382 24149 raft_consensus.cc:729] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:49.874498 24149 consensus_queue.cc:260] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:49.874728 24149 ts_tablet_manager.cc:1398] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:49.874892 24149 tablet_bootstrap.cc:492] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:49.875614 24149 tablet_bootstrap.cc:654] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:49.876412 24145 tablet_bootstrap.cc:492] T af902251d1624f95893f9bb29795d126 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:49.876484 24145 ts_tablet_manager.cc:1367] T af902251d1624f95893f9bb29795d126 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:49.876847 24145 raft_consensus.cc:348] T af902251d1624f95893f9bb29795d126 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:49.876950 24145 raft_consensus.cc:374] T af902251d1624f95893f9bb29795d126 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:49.876984 24145 raft_consensus.cc:729] T af902251d1624f95893f9bb29795d126 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:49.877128 24145 consensus_queue.cc:260] T af902251d1624f95893f9bb29795d126 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:49.877405 24145 ts_tablet_manager.cc:1398] T af902251d1624f95893f9bb29795d126 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:49.877833 24149 tablet_bootstrap.cc:492] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:49.877902 24149 ts_tablet_manager.cc:1367] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:49.878187 24149 raft_consensus.cc:348] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:49.878266 24149 raft_consensus.cc:374] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:49.878288 24149 raft_consensus.cc:729] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:49.878383 24149 consensus_queue.cc:260] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:49.878620 24149 ts_tablet_manager.cc:1398] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:49.878764 24149 tablet_bootstrap.cc:492] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:49.879473 24149 tablet_bootstrap.cc:654] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:49.880030 24145 tablet_bootstrap.cc:492] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:49.881902 24145 tablet_bootstrap.cc:654] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:49.882746 24149 tablet_bootstrap.cc:492] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:49.882850 24149 ts_tablet_manager.cc:1367] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:49.882911 24145 tablet_bootstrap.cc:492] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:49.882994 24145 ts_tablet_manager.cc:1367] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:49.883268 24149 raft_consensus.cc:348] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.883360 24149 raft_consensus.cc:374] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:49.883395 24149 raft_consensus.cc:729] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:49.883450 24145 raft_consensus.cc:348] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:49.883569 24145 raft_consensus.cc:374] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:49.883530 24149 consensus_queue.cc:260] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.883610 24145 raft_consensus.cc:729] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:49.883729 24145 consensus_queue.cc:260] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:49.883822 24149 ts_tablet_manager.cc:1398] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:49.883996 24145 ts_tablet_manager.cc:1398] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:49.899148 23853 raft_consensus.cc:482] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:49.899264 23853 raft_consensus.cc:504] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.899703 23853 leader_election.cc:288] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:49.900077 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "423d6be23b044130b0c916f995dffced" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:49.900146 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "423d6be23b044130b0c916f995dffced" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:49.900218 22634 raft_consensus.cc:2454] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:49.900280 22765 raft_consensus.cc:2454] T 423d6be23b044130b0c916f995dffced P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:49.900566 22308 leader_election.cc:302] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:49.900789 23853 raft_consensus.cc:2781] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:49.900876 23853 raft_consensus.cc:482] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:49.900929 23853 raft_consensus.cc:3037] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:49.902004 23853 raft_consensus.cc:504] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.902307 23853 leader_election.cc:288] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:49.902585 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "423d6be23b044130b0c916f995dffced" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:49.902683 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "423d6be23b044130b0c916f995dffced" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:49.902740 22634 raft_consensus.cc:3037] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:49.902804 22765 raft_consensus.cc:3037] T 423d6be23b044130b0c916f995dffced P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:49.903475 22634 raft_consensus.cc:2454] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:49.903684 22765 raft_consensus.cc:2454] T 423d6be23b044130b0c916f995dffced P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:49.903777 22308 leader_election.cc:302] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:49.904057 23853 raft_consensus.cc:2781] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:49.904143 23853 raft_consensus.cc:686] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:58:49.904295 23853 consensus_queue.cc:237] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:49.906237 22221 catalog_manager.cc:5484] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:58:49.996660 24144 raft_consensus.cc:482] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:49.996847 24144 raft_consensus.cc:504] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:49.997390 24144 leader_election.cc:288] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:49.997848 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2241a741bf85433b9fea512e9c530416" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:49.997867 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2241a741bf85433b9fea512e9c530416" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:49.997999 22765 raft_consensus.cc:2454] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:49.998021 22502 raft_consensus.cc:2454] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:49.998340 22567 leader_election.cc:302] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:49.998495 24144 raft_consensus.cc:2781] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:49.998558 24144 raft_consensus.cc:482] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:49.998596 24144 raft_consensus.cc:3037] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:49.999387 24144 raft_consensus.cc:504] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:49.999696 24144 leader_election.cc:288] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:50.000080 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2241a741bf85433b9fea512e9c530416" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:50.000123 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2241a741bf85433b9fea512e9c530416" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:50.000196 22502 raft_consensus.cc:3037] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:50.000226 22765 raft_consensus.cc:3037] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:50.000929 22502 raft_consensus.cc:2454] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:50.000979 22765 raft_consensus.cc:2454] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:50.001260 22570 leader_election.cc:302] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:50.001446 24144 raft_consensus.cc:2781] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:50.001538 24144 raft_consensus.cc:686] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:50.001703 24144 consensus_queue.cc:237] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:50.003582 22221 catalog_manager.cc:5484] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:58:50.084496 24144 raft_consensus.cc:482] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:50.084661 24144 raft_consensus.cc:504] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:50.085130 24144 leader_election.cc:288] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:50.085615 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af902251d1624f95893f9bb29795d126" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:50.085680 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af902251d1624f95893f9bb29795d126" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:50.085765 22372 raft_consensus.cc:2454] T af902251d1624f95893f9bb29795d126 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:50.085834 22502 raft_consensus.cc:2454] T af902251d1624f95893f9bb29795d126 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:50.086200 22567 leader_election.cc:302] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:50.086378 24144 raft_consensus.cc:2781] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:50.086426 24144 raft_consensus.cc:482] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:50.086470 24144 raft_consensus.cc:3037] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:50.087484 24144 raft_consensus.cc:504] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:50.087862 24144 leader_election.cc:288] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:50.088182 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af902251d1624f95893f9bb29795d126" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:50.088148 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af902251d1624f95893f9bb29795d126" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:50.088285 22372 raft_consensus.cc:3037] T af902251d1624f95893f9bb29795d126 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:50.088294 22502 raft_consensus.cc:3037] T af902251d1624f95893f9bb29795d126 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:50.089007 22502 raft_consensus.cc:2454] T af902251d1624f95893f9bb29795d126 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:50.089253 22372 raft_consensus.cc:2454] T af902251d1624f95893f9bb29795d126 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:50.089252 22567 leader_election.cc:302] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:50.089449 24144 raft_consensus.cc:2781] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:50.089532 24144 raft_consensus.cc:686] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:50.089669 24144 consensus_queue.cc:237] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:50.091626 22221 catalog_manager.cc:5484] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
W20240502 07:58:50.211213 24160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:50.211473 24160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:50.215075 24160 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_11'
I20240502 07:58:50.303521 22372 raft_consensus.cc:1261] T af902251d1624f95893f9bb29795d126 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:50.303521 22502 raft_consensus.cc:1261] T af902251d1624f95893f9bb29795d126 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:50.304273 22765 raft_consensus.cc:1261] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:50.304345 24073 consensus_queue.cc:1035] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:50.305471 22502 raft_consensus.cc:1261] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:50.308956 24144 consensus_queue.cc:1035] T af902251d1624f95893f9bb29795d126 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:50.309293 24144 consensus_queue.cc:1035] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:50.309640 22765 raft_consensus.cc:1261] T 423d6be23b044130b0c916f995dffced P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:50.310035 22634 raft_consensus.cc:1261] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:50.310477 24041 consensus_queue.cc:1035] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:50.310848 24041 consensus_queue.cc:1035] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:50.314648 24192 consensus_queue.cc:1035] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 116.536 ms
time per row: 0.0582682 ms
I20240502 07:58:50.376087 22230 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:50.382030 22351 tablet_service.cc:1467] Processing CreateTablet for tablet 257f0f4dca624c859d3b06cfd4246536 (DEFAULT_TABLE table=rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:50.382360 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 257f0f4dca624c859d3b06cfd4246536. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:50.382385 22482 tablet_service.cc:1467] Processing CreateTablet for tablet 257f0f4dca624c859d3b06cfd4246536 (DEFAULT_TABLE table=rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:50.382438 22483 tablet_service.cc:1467] Processing CreateTablet for tablet f9cf9213d36b42238e9d0c1c4c8a840a (DEFAULT_TABLE table=rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:50.382640 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 257f0f4dca624c859d3b06cfd4246536. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:50.382778 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9cf9213d36b42238e9d0c1c4c8a840a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:50.383634 22743 tablet_service.cc:1467] Processing CreateTablet for tablet f9cf9213d36b42238e9d0c1c4c8a840a (DEFAULT_TABLE table=rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:50.383867 22350 tablet_service.cc:1467] Processing CreateTablet for tablet f9cf9213d36b42238e9d0c1c4c8a840a (DEFAULT_TABLE table=rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:50.383934 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9cf9213d36b42238e9d0c1c4c8a840a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:50.384099 22350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9cf9213d36b42238e9d0c1c4c8a840a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:50.384899 22614 tablet_service.cc:1467] Processing CreateTablet for tablet 257f0f4dca624c859d3b06cfd4246536 (DEFAULT_TABLE table=rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:50.385185 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 257f0f4dca624c859d3b06cfd4246536. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:50.385452 24145 tablet_bootstrap.cc:492] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:50.386390 24145 tablet_bootstrap.cc:654] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:50.387547 24145 tablet_bootstrap.cc:492] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:50.387511 22744 tablet_service.cc:1467] Processing CreateTablet for tablet 91f8c025d982409db2a1198a3ddf5837 (DEFAULT_TABLE table=rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:50.387607 24145 ts_tablet_manager.cc:1367] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:50.387737 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91f8c025d982409db2a1198a3ddf5837. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:50.387959 24145 raft_consensus.cc:348] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:50.388041 24145 raft_consensus.cc:374] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:50.388062 24145 raft_consensus.cc:729] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:50.388190 24145 consensus_queue.cc:260] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:50.388450 24145 ts_tablet_manager.cc:1398] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:50.388576 24145 tablet_bootstrap.cc:492] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:50.388880 22614 tablet_service.cc:1467] Processing CreateTablet for tablet 91f8c025d982409db2a1198a3ddf5837 (DEFAULT_TABLE table=rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:50.389031 24208 tablet_bootstrap.cc:492] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:50.389137 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91f8c025d982409db2a1198a3ddf5837. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:50.389470 24145 tablet_bootstrap.cc:654] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:50.389607 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 91f8c025d982409db2a1198a3ddf5837 (DEFAULT_TABLE table=rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:50.389861 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91f8c025d982409db2a1198a3ddf5837. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:50.389910 24208 tablet_bootstrap.cc:654] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:50.390956 24208 tablet_bootstrap.cc:492] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:50.391036 24208 ts_tablet_manager.cc:1367] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:50.391455 24208 raft_consensus.cc:348] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:50.391553 24208 raft_consensus.cc:374] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:50.391592 24208 raft_consensus.cc:729] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:50.391718 24208 consensus_queue.cc:260] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:50.392005 24208 ts_tablet_manager.cc:1398] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:50.392058 24149 tablet_bootstrap.cc:492] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:50.392866 24149 tablet_bootstrap.cc:654] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:50.393911 24149 tablet_bootstrap.cc:492] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:50.394013 24149 ts_tablet_manager.cc:1367] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:50.394237 24208 tablet_bootstrap.cc:492] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:50.394424 24207 tablet_bootstrap.cc:492] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:50.394490 24149 raft_consensus.cc:348] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:50.394599 24149 raft_consensus.cc:374] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:50.394645 24149 raft_consensus.cc:729] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:50.394766 24149 consensus_queue.cc:260] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:50.395059 24149 ts_tablet_manager.cc:1398] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:50.395176 24208 tablet_bootstrap.cc:654] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:50.395220 24149 tablet_bootstrap.cc:492] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:50.395255 24207 tablet_bootstrap.cc:654] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:50.396068 24149 tablet_bootstrap.cc:654] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:50.396682 24145 tablet_bootstrap.cc:492] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:50.396785 24145 ts_tablet_manager.cc:1367] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 07:58:50.397261 24145 raft_consensus.cc:348] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:50.397414 24145 raft_consensus.cc:374] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:50.397450 24145 raft_consensus.cc:729] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:50.397616 24149 tablet_bootstrap.cc:492] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:50.397579 24145 consensus_queue.cc:260] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:50.397701 24149 ts_tablet_manager.cc:1367] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:50.397887 24145 ts_tablet_manager.cc:1398] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:50.398152 24149 raft_consensus.cc:348] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:50.398255 24149 raft_consensus.cc:374] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:50.398293 24149 raft_consensus.cc:729] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:50.398430 24149 consensus_queue.cc:260] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:50.398682 24149 ts_tablet_manager.cc:1398] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:50.400557 24207 tablet_bootstrap.cc:492] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:50.400882 24207 ts_tablet_manager.cc:1367] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:58:50.400941 24208 tablet_bootstrap.cc:492] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:50.401019 24208 ts_tablet_manager.cc:1367] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:58:50.401505 24208 raft_consensus.cc:348] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:50.401639 24208 raft_consensus.cc:374] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:50.401682 24208 raft_consensus.cc:729] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:50.401755 24207 raft_consensus.cc:348] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:50.401796 24208 consensus_queue.cc:260] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:50.401860 24207 raft_consensus.cc:374] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:50.401898 24207 raft_consensus.cc:729] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:50.402105 24208 ts_tablet_manager.cc:1398] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:50.402072 24207 consensus_queue.cc:260] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:50.402398 24207 ts_tablet_manager.cc:1398] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:50.402670 24207 tablet_bootstrap.cc:492] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:50.403630 24207 tablet_bootstrap.cc:654] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:50.404987 24207 tablet_bootstrap.cc:492] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:50.405112 24207 ts_tablet_manager.cc:1367] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:50.405900 24207 raft_consensus.cc:348] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:50.405992 24207 raft_consensus.cc:374] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:50.406028 24207 raft_consensus.cc:729] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:50.406162 24207 consensus_queue.cc:260] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:50.406471 24207 ts_tablet_manager.cc:1398] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:50.406643 24207 tablet_bootstrap.cc:492] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:50.407528 24207 tablet_bootstrap.cc:654] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:50.408465 24207 tablet_bootstrap.cc:492] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:50.408545 24207 ts_tablet_manager.cc:1367] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 07:58:50.408936 24207 raft_consensus.cc:348] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:50.409050 24207 raft_consensus.cc:374] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:50.409086 24207 raft_consensus.cc:729] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:50.409255 24207 consensus_queue.cc:260] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:50.409621 24207 ts_tablet_manager.cc:1398] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:50.495992 24199 raft_consensus.cc:482] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:50.496109 24199 raft_consensus.cc:504] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:50.496488 24199 leader_election.cc:288] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:50.496791 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f8c025d982409db2a1198a3ddf5837" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:50.496906 22765 raft_consensus.cc:2454] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:50.496876 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f8c025d982409db2a1198a3ddf5837" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:50.497036 22634 raft_consensus.cc:2454] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:50.497238 22308 leader_election.cc:302] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:50.497431 24199 raft_consensus.cc:2781] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:50.497493 24199 raft_consensus.cc:482] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:50.497542 24199 raft_consensus.cc:3037] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:50.498406 24199 raft_consensus.cc:504] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:50.498713 24199 leader_election.cc:288] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:50.498988 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f8c025d982409db2a1198a3ddf5837" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:50.499042 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f8c025d982409db2a1198a3ddf5837" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:50.499099 22765 raft_consensus.cc:3037] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:50.499181 22634 raft_consensus.cc:3037] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:50.499841 22765 raft_consensus.cc:2454] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:50.499850 22634 raft_consensus.cc:2454] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:50.500118 22308 leader_election.cc:302] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:50.500284 24199 raft_consensus.cc:2781] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:50.500375 24199 raft_consensus.cc:686] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:58:50.500535 24199 consensus_queue.cc:237] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:50.502283 22230 catalog_manager.cc:5484] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:58:50.516517 24073 raft_consensus.cc:482] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:50.516638 24073 raft_consensus.cc:504] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:50.517117 24073 leader_election.cc:288] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:50.517514 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "257f0f4dca624c859d3b06cfd4246536" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:50.517585 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "257f0f4dca624c859d3b06cfd4246536" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:50.517648 22372 raft_consensus.cc:2454] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:50.517728 22501 raft_consensus.cc:2454] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:50.517983 22570 leader_election.cc:302] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:50.518195 24073 raft_consensus.cc:2781] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:50.518257 24073 raft_consensus.cc:482] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:50.518307 24073 raft_consensus.cc:3037] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:50.519142 24073 raft_consensus.cc:504] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:50.519462 24073 leader_election.cc:288] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:50.519821 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "257f0f4dca624c859d3b06cfd4246536" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:50.519837 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "257f0f4dca624c859d3b06cfd4246536" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:50.519927 22372 raft_consensus.cc:3037] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:50.519946 22501 raft_consensus.cc:3037] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:50.520684 22372 raft_consensus.cc:2454] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:50.520956 22570 leader_election.cc:302] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:50.521000 22501 raft_consensus.cc:2454] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:50.521118 24073 raft_consensus.cc:2781] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:50.521209 24073 raft_consensus.cc:686] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:50.521385 24073 consensus_queue.cc:237] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:50.523293 22230 catalog_manager.cc:5484] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:58:50.702288 24199 raft_consensus.cc:482] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:50.702414 24199 raft_consensus.cc:504] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:50.702841 24199 leader_election.cc:288] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:50.703248 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9cf9213d36b42238e9d0c1c4c8a840a" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:50.703267 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9cf9213d36b42238e9d0c1c4c8a840a" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:50.703389 22501 raft_consensus.cc:2454] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:50.703398 22765 raft_consensus.cc:2454] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:50.703675 22305 leader_election.cc:302] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:50.703860 24199 raft_consensus.cc:2781] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:50.703918 24199 raft_consensus.cc:482] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:50.703945 24199 raft_consensus.cc:3037] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:50.704813 24199 raft_consensus.cc:504] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:50.705138 24199 leader_election.cc:288] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:50.705484 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9cf9213d36b42238e9d0c1c4c8a840a" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:50.705598 22765 raft_consensus.cc:3037] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:50.706518 22765 raft_consensus.cc:2454] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:50.706758 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9cf9213d36b42238e9d0c1c4c8a840a" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:50.706801 22308 leader_election.cc:302] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:50.706887 22501 raft_consensus.cc:3037] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:50.706965 24199 raft_consensus.cc:2781] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:50.707120 24199 raft_consensus.cc:686] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:58:50.707257 24199 consensus_queue.cc:237] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:50.707736 22501 raft_consensus.cc:2454] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:50.709461 22230 catalog_manager.cc:5484] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
W20240502 07:58:50.870471 24221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:50.870692 24221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:50.873785 24221 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_12'
I20240502 07:58:50.956065 22634 raft_consensus.cc:1261] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:50.956542 22763 raft_consensus.cc:1261] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:50.956540 23853 consensus_queue.cc:1035] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:50.957021 23853 consensus_queue.cc:1035] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:50.963197 22765 raft_consensus.cc:1261] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:50.964164 24249 mvcc.cc:205] Tried to move back new op lower bound from 7023152049992327168 to 7023152048131354624. Current Snapshot: MvccSnapshot[applied={T|T < 7023152049992327168}]
I20240502 07:58:50.966171 24199 consensus_queue.cc:1035] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:50.967866 22501 raft_consensus.cc:1261] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:50.968300 24199 consensus_queue.cc:1035] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:50.977027 22502 raft_consensus.cc:1261] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:50.977535 24073 consensus_queue.cc:1035] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:50.978583 22370 raft_consensus.cc:1261] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:50.979040 24073 consensus_queue.cc:1035] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 104.51 ms
time per row: 0.052255 ms
I20240502 07:58:51.029559 22227 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_13"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:51.035699 22482 tablet_service.cc:1467] Processing CreateTablet for tablet 72964b1c935441ca840a46ff79921cc3 (DEFAULT_TABLE table=rebalance_test_table_13 [id=490b2fcdef13484180791ae6a0d1907d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:51.036006 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72964b1c935441ca840a46ff79921cc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:51.036628 22614 tablet_service.cc:1467] Processing CreateTablet for tablet 72964b1c935441ca840a46ff79921cc3 (DEFAULT_TABLE table=rebalance_test_table_13 [id=490b2fcdef13484180791ae6a0d1907d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:51.036897 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72964b1c935441ca840a46ff79921cc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:51.036851 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 17421eec61584fbcac13f441922269bc (DEFAULT_TABLE table=rebalance_test_table_13 [id=490b2fcdef13484180791ae6a0d1907d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:51.036851 22744 tablet_service.cc:1467] Processing CreateTablet for tablet 72964b1c935441ca840a46ff79921cc3 (DEFAULT_TABLE table=rebalance_test_table_13 [id=490b2fcdef13484180791ae6a0d1907d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:51.037199 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72964b1c935441ca840a46ff79921cc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:51.037793 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 02449bcab2294956918d1889485cfd5f (DEFAULT_TABLE table=rebalance_test_table_13 [id=490b2fcdef13484180791ae6a0d1907d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:51.038040 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02449bcab2294956918d1889485cfd5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:51.038615 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 02449bcab2294956918d1889485cfd5f (DEFAULT_TABLE table=rebalance_test_table_13 [id=490b2fcdef13484180791ae6a0d1907d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:51.038844 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02449bcab2294956918d1889485cfd5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:51.039325 22613 tablet_service.cc:1467] Processing CreateTablet for tablet 17421eec61584fbcac13f441922269bc (DEFAULT_TABLE table=rebalance_test_table_13 [id=490b2fcdef13484180791ae6a0d1907d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:51.039533 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17421eec61584fbcac13f441922269bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:51.040300 24281 tablet_bootstrap.cc:492] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:51.040558 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17421eec61584fbcac13f441922269bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:51.040985 22482 tablet_service.cc:1467] Processing CreateTablet for tablet 17421eec61584fbcac13f441922269bc (DEFAULT_TABLE table=rebalance_test_table_13 [id=490b2fcdef13484180791ae6a0d1907d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:51.041208 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17421eec61584fbcac13f441922269bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:51.041652 22612 tablet_service.cc:1467] Processing CreateTablet for tablet 02449bcab2294956918d1889485cfd5f (DEFAULT_TABLE table=rebalance_test_table_13 [id=490b2fcdef13484180791ae6a0d1907d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:51.041891 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02449bcab2294956918d1889485cfd5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:51.042917 24278 tablet_bootstrap.cc:492] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:51.043182 24281 tablet_bootstrap.cc:654] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:51.043198 24279 tablet_bootstrap.cc:492] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:51.043782 24278 tablet_bootstrap.cc:654] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:51.044095 24279 tablet_bootstrap.cc:654] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:51.044474 24280 tablet_bootstrap.cc:492] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:51.045162 24281 tablet_bootstrap.cc:492] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:51.045248 24281 ts_tablet_manager.cc:1367] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:51.045449 24279 tablet_bootstrap.cc:492] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:51.045614 24280 tablet_bootstrap.cc:654] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:51.045625 24279 ts_tablet_manager.cc:1367] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:51.045782 24281 raft_consensus.cc:348] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.045890 24278 tablet_bootstrap.cc:492] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:51.045966 24278 ts_tablet_manager.cc:1367] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:51.046022 24281 raft_consensus.cc:374] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:51.046099 24279 raft_consensus.cc:348] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:51.046157 24281 raft_consensus.cc:729] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:51.046205 24279 raft_consensus.cc:374] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:51.046329 24279 raft_consensus.cc:729] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:51.046346 24281 consensus_queue.cc:260] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.046499 24279 consensus_queue.cc:260] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:51.046504 24278 raft_consensus.cc:348] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:51.046598 24278 raft_consensus.cc:374] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:51.046630 24278 raft_consensus.cc:729] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:51.046669 24281 ts_tablet_manager.cc:1398] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:51.046746 24278 consensus_queue.cc:260] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:51.046984 24279 ts_tablet_manager.cc:1398] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:51.047049 24278 ts_tablet_manager.cc:1398] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:51.047073 24280 tablet_bootstrap.cc:492] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:51.047163 24279 tablet_bootstrap.cc:492] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:51.047158 24280 ts_tablet_manager.cc:1367] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:51.047223 24278 tablet_bootstrap.cc:492] T 02449bcab2294956918d1889485cfd5f P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:51.047636 24280 raft_consensus.cc:348] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:51.047757 24280 raft_consensus.cc:374] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:51.047792 24280 raft_consensus.cc:729] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:51.047940 24278 tablet_bootstrap.cc:654] T 02449bcab2294956918d1889485cfd5f P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:51.047933 24280 consensus_queue.cc:260] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:51.048113 24279 tablet_bootstrap.cc:654] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:51.048312 24280 ts_tablet_manager.cc:1398] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:51.048497 24280 tablet_bootstrap.cc:492] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:51.049288 24279 tablet_bootstrap.cc:492] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:51.049499 24280 tablet_bootstrap.cc:654] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:51.049491 24279 ts_tablet_manager.cc:1367] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.001s
I20240502 07:58:51.049997 24279 raft_consensus.cc:348] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:51.050096 24279 raft_consensus.cc:374] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:51.050137 24279 raft_consensus.cc:729] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:51.050290 24279 consensus_queue.cc:260] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:51.050665 24279 ts_tablet_manager.cc:1398] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:51.050873 24279 tablet_bootstrap.cc:492] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:51.051865 24278 tablet_bootstrap.cc:492] T 02449bcab2294956918d1889485cfd5f P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:51.051905 24279 tablet_bootstrap.cc:654] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:51.051962 24278 ts_tablet_manager.cc:1367] T 02449bcab2294956918d1889485cfd5f P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:51.052381 24278 raft_consensus.cc:348] T 02449bcab2294956918d1889485cfd5f P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.052459 24278 raft_consensus.cc:374] T 02449bcab2294956918d1889485cfd5f P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:51.052477 24278 raft_consensus.cc:729] T 02449bcab2294956918d1889485cfd5f P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:51.052577 24278 consensus_queue.cc:260] T 02449bcab2294956918d1889485cfd5f P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.052860 24278 ts_tablet_manager.cc:1398] T 02449bcab2294956918d1889485cfd5f P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:51.053025 24278 tablet_bootstrap.cc:492] T 17421eec61584fbcac13f441922269bc P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:51.053225 24279 tablet_bootstrap.cc:492] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:51.053292 24279 ts_tablet_manager.cc:1367] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:51.053496 24280 tablet_bootstrap.cc:492] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:51.053571 24280 ts_tablet_manager.cc:1367] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:51.053719 24279 raft_consensus.cc:348] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.053800 24279 raft_consensus.cc:374] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:51.053824 24279 raft_consensus.cc:729] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:51.053943 24279 consensus_queue.cc:260] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.053974 24280 raft_consensus.cc:348] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:51.054071 24280 raft_consensus.cc:374] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:51.054102 24280 raft_consensus.cc:729] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:51.054205 24279 ts_tablet_manager.cc:1398] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:51.054212 24280 consensus_queue.cc:260] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:51.054498 24280 ts_tablet_manager.cc:1398] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:51.055158 24278 tablet_bootstrap.cc:654] T 17421eec61584fbcac13f441922269bc P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:51.056363 24278 tablet_bootstrap.cc:492] T 17421eec61584fbcac13f441922269bc P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:51.056589 24278 ts_tablet_manager.cc:1367] T 17421eec61584fbcac13f441922269bc P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:51.057106 24278 raft_consensus.cc:348] T 17421eec61584fbcac13f441922269bc P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:51.057224 24278 raft_consensus.cc:374] T 17421eec61584fbcac13f441922269bc P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:51.057302 24278 raft_consensus.cc:729] T 17421eec61584fbcac13f441922269bc P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:51.057482 24278 consensus_queue.cc:260] T 17421eec61584fbcac13f441922269bc P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:51.058318 24278 ts_tablet_manager.cc:1398] T 17421eec61584fbcac13f441922269bc P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:51.070475 24266 raft_consensus.cc:482] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:51.070596 24266 raft_consensus.cc:504] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.071038 24266 leader_election.cc:288] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:51.071406 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02449bcab2294956918d1889485cfd5f" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:51.071569 22634 raft_consensus.cc:2454] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:51.071607 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02449bcab2294956918d1889485cfd5f" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:51.071820 22502 raft_consensus.cc:2454] T 02449bcab2294956918d1889485cfd5f P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:51.071942 22308 leader_election.cc:302] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:51.072146 24266 raft_consensus.cc:2781] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:51.072203 24266 raft_consensus.cc:482] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:51.072263 24266 raft_consensus.cc:3037] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:51.073207 24266 raft_consensus.cc:504] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.073596 24266 leader_election.cc:288] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:51.073941 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02449bcab2294956918d1889485cfd5f" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:51.073967 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02449bcab2294956918d1889485cfd5f" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:51.074082 22634 raft_consensus.cc:3037] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:51.074082 22502 raft_consensus.cc:3037] T 02449bcab2294956918d1889485cfd5f P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:51.075037 22502 raft_consensus.cc:2454] T 02449bcab2294956918d1889485cfd5f P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:51.075037 22634 raft_consensus.cc:2454] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:51.075333 22308 leader_election.cc:302] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:51.075570 24266 raft_consensus.cc:2781] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:51.075652 24266 raft_consensus.cc:686] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:58:51.075801 24266 consensus_queue.cc:237] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.078176 22230 catalog_manager.cc:5484] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:58:51.080116 24073 raft_consensus.cc:482] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:51.080205 24073 raft_consensus.cc:504] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:51.080596 24073 leader_election.cc:288] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:51.080911 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17421eec61584fbcac13f441922269bc" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:51.080930 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17421eec61584fbcac13f441922269bc" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:51.081027 22502 raft_consensus.cc:2454] T 17421eec61584fbcac13f441922269bc P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:51.081045 22763 raft_consensus.cc:2454] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:51.081352 22570 leader_election.cc:302] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:51.081526 24073 raft_consensus.cc:2781] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:51.081583 24073 raft_consensus.cc:482] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:51.081648 24073 raft_consensus.cc:3037] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:51.082648 24073 raft_consensus.cc:504] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:51.082943 24073 leader_election.cc:288] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:51.083235 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17421eec61584fbcac13f441922269bc" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:51.083333 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17421eec61584fbcac13f441922269bc" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:51.083380 22763 raft_consensus.cc:3037] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:51.083447 22502 raft_consensus.cc:3037] T 17421eec61584fbcac13f441922269bc P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:51.084132 22502 raft_consensus.cc:2454] T 17421eec61584fbcac13f441922269bc P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:51.084283 22763 raft_consensus.cc:2454] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:51.084378 22567 leader_election.cc:302] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:51.084663 24073 raft_consensus.cc:2781] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:51.084745 24073 raft_consensus.cc:686] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:51.084874 24073 consensus_queue.cc:237] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:51.086578 22227 catalog_manager.cc:5484] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:58:51.220037 24073 raft_consensus.cc:482] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:51.220180 24073 raft_consensus.cc:504] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:51.220564 24073 leader_election.cc:288] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:51.220958 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72964b1c935441ca840a46ff79921cc3" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:51.220957 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72964b1c935441ca840a46ff79921cc3" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:51.221094 22763 raft_consensus.cc:2454] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:51.221107 22502 raft_consensus.cc:2454] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:51.221405 22570 leader_election.cc:302] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:51.221563 24073 raft_consensus.cc:2781] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:51.221624 24073 raft_consensus.cc:482] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:51.221657 24073 raft_consensus.cc:3037] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:51.222460 24073 raft_consensus.cc:504] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:51.222775 24073 leader_election.cc:288] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:51.223136 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72964b1c935441ca840a46ff79921cc3" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:51.223136 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72964b1c935441ca840a46ff79921cc3" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:51.223258 22502 raft_consensus.cc:3037] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:51.223258 22763 raft_consensus.cc:3037] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:51.224236 22763 raft_consensus.cc:2454] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:51.224236 22502 raft_consensus.cc:2454] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:51.224543 22570 leader_election.cc:302] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:51.224699 24073 raft_consensus.cc:2781] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:51.224797 24073 raft_consensus.cc:686] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:51.224941 24073 consensus_queue.cc:237] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:51.226732 22227 catalog_manager.cc:5484] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
W20240502 07:58:51.352469 24294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:51.352710 24294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:51.355792 24294 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_13'
I20240502 07:58:51.434897 22502 raft_consensus.cc:1261] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:51.435319 24276 consensus_queue.cc:1035] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:51.435853 22763 raft_consensus.cc:1261] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:51.436434 24192 consensus_queue.cc:1035] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:51.437017 22763 raft_consensus.cc:1261] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:51.437448 24073 consensus_queue.cc:1035] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:51.438829 22634 raft_consensus.cc:1261] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:51.439328 24266 consensus_queue.cc:1035] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:51.443826 22501 raft_consensus.cc:1261] T 02449bcab2294956918d1889485cfd5f P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:51.444643 22501 raft_consensus.cc:1261] T 17421eec61584fbcac13f441922269bc P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:51.455601 24073 consensus_queue.cc:1035] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:51.456332 23853 consensus_queue.cc:1035] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 121.595 ms
time per row: 0.0607977 ms
I20240502 07:58:51.519217 22225 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:51.525498 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 7b935ed00d34452fb6c9d528d5f67cc5 (DEFAULT_TABLE table=rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:51.525849 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b935ed00d34452fb6c9d528d5f67cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:51.526279 22350 tablet_service.cc:1467] Processing CreateTablet for tablet d6dfeeb6b16a4f92bc756bf8933c7fb1 (DEFAULT_TABLE table=rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:51.526319 22351 tablet_service.cc:1467] Processing CreateTablet for tablet ef6c248c0e15401b9eceeb6a428193dc (DEFAULT_TABLE table=rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:51.526506 22350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6dfeeb6b16a4f92bc756bf8933c7fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:51.526765 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef6c248c0e15401b9eceeb6a428193dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:51.528275 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 7b935ed00d34452fb6c9d528d5f67cc5 (DEFAULT_TABLE table=rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:51.528499 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b935ed00d34452fb6c9d528d5f67cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:51.528527 24280 tablet_bootstrap.cc:492] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:51.528846 24281 tablet_bootstrap.cc:492] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:51.529397 24280 tablet_bootstrap.cc:654] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:51.529712 24281 tablet_bootstrap.cc:654] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:51.530577 24280 tablet_bootstrap.cc:492] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:51.530660 24280 ts_tablet_manager.cc:1367] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:58:51.530903 22744 tablet_service.cc:1467] Processing CreateTablet for tablet ef6c248c0e15401b9eceeb6a428193dc (DEFAULT_TABLE table=rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:51.531124 24280 raft_consensus.cc:348] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.531235 24280 raft_consensus.cc:374] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:51.531184 22612 tablet_service.cc:1467] Processing CreateTablet for tablet d6dfeeb6b16a4f92bc756bf8933c7fb1 (DEFAULT_TABLE table=rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:51.531273 24280 raft_consensus.cc:729] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:51.531395 24280 consensus_queue.cc:260] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.531455 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6dfeeb6b16a4f92bc756bf8933c7fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:51.531697 24280 ts_tablet_manager.cc:1398] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:51.531865 22482 tablet_service.cc:1467] Processing CreateTablet for tablet 7b935ed00d34452fb6c9d528d5f67cc5 (DEFAULT_TABLE table=rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:51.532146 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b935ed00d34452fb6c9d528d5f67cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:51.532217 22743 tablet_service.cc:1467] Processing CreateTablet for tablet d6dfeeb6b16a4f92bc756bf8933c7fb1 (DEFAULT_TABLE table=rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:51.531121 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef6c248c0e15401b9eceeb6a428193dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:51.532545 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6dfeeb6b16a4f92bc756bf8933c7fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:51.534354 24280 tablet_bootstrap.cc:492] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:51.534689 22483 tablet_service.cc:1467] Processing CreateTablet for tablet ef6c248c0e15401b9eceeb6a428193dc (DEFAULT_TABLE table=rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:51.534965 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef6c248c0e15401b9eceeb6a428193dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:51.535210 24280 tablet_bootstrap.cc:654] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:51.535593 24279 tablet_bootstrap.cc:492] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:51.536553 24280 tablet_bootstrap.cc:492] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:51.536608 24280 ts_tablet_manager.cc:1367] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:51.536613 24279 tablet_bootstrap.cc:654] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:51.536916 24280 raft_consensus.cc:348] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:51.536986 24280 raft_consensus.cc:374] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:51.537006 24280 raft_consensus.cc:729] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:51.537101 24280 consensus_queue.cc:260] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:51.537189 24278 tablet_bootstrap.cc:492] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:51.537395 24280 ts_tablet_manager.cc:1398] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:51.537549 24280 tablet_bootstrap.cc:492] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:51.538125 24278 tablet_bootstrap.cc:654] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:51.538183 24279 tablet_bootstrap.cc:492] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:51.538267 24279 ts_tablet_manager.cc:1367] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:51.538393 24280 tablet_bootstrap.cc:654] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:51.538762 24279 raft_consensus.cc:348] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:51.538880 24279 raft_consensus.cc:374] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:51.538934 24279 raft_consensus.cc:729] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:51.539072 24279 consensus_queue.cc:260] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:51.539347 24279 ts_tablet_manager.cc:1398] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:51.539386 24281 tablet_bootstrap.cc:492] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:51.539471 24281 ts_tablet_manager.cc:1367] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 07:58:51.539929 24281 raft_consensus.cc:348] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.540035 24281 raft_consensus.cc:374] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:51.540066 24281 raft_consensus.cc:729] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:51.540170 24281 consensus_queue.cc:260] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.540434 24281 ts_tablet_manager.cc:1398] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:51.540601 24281 tablet_bootstrap.cc:492] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:51.541817 24281 tablet_bootstrap.cc:654] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:51.542860 24281 tablet_bootstrap.cc:492] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:51.542939 24281 ts_tablet_manager.cc:1367] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:51.543325 24281 raft_consensus.cc:348] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:51.543409 24281 raft_consensus.cc:374] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:51.543440 24281 raft_consensus.cc:729] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:51.543548 24281 consensus_queue.cc:260] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:51.543745 24281 ts_tablet_manager.cc:1398] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:51.543879 24281 tablet_bootstrap.cc:492] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:51.544371 24278 tablet_bootstrap.cc:492] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:51.544454 24278 ts_tablet_manager.cc:1367] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:58:51.544596 24281 tablet_bootstrap.cc:654] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:51.544922 24278 raft_consensus.cc:348] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.545060 24278 raft_consensus.cc:374] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:51.545102 24278 raft_consensus.cc:729] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:51.545251 24278 consensus_queue.cc:260] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.545675 24278 ts_tablet_manager.cc:1398] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:51.545745 24280 tablet_bootstrap.cc:492] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:51.545823 24280 ts_tablet_manager.cc:1367] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:58:51.545846 24278 tablet_bootstrap.cc:492] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:51.545931 24281 tablet_bootstrap.cc:492] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:51.545993 24281 ts_tablet_manager.cc:1367] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:51.546237 24280 raft_consensus.cc:348] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.546331 24280 raft_consensus.cc:374] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:51.546365 24280 raft_consensus.cc:729] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:51.546329 24281 raft_consensus.cc:348] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.546412 24281 raft_consensus.cc:374] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:51.546447 24281 raft_consensus.cc:729] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:51.546478 24280 consensus_queue.cc:260] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.546573 24281 consensus_queue.cc:260] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.546747 24280 ts_tablet_manager.cc:1398] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:51.547044 24278 tablet_bootstrap.cc:654] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:51.546808 24281 ts_tablet_manager.cc:1398] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:51.549245 24278 tablet_bootstrap.cc:492] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:51.549444 24278 ts_tablet_manager.cc:1367] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:51.549928 24278 raft_consensus.cc:348] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.550051 24278 raft_consensus.cc:374] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:51.550133 24278 raft_consensus.cc:729] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:51.550385 24278 consensus_queue.cc:260] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.550697 24278 ts_tablet_manager.cc:1398] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:51.582764 24337 raft_consensus.cc:482] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:51.582914 24337 raft_consensus.cc:504] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:51.583343 24337 leader_election.cc:288] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:51.583828 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6dfeeb6b16a4f92bc756bf8933c7fb1" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:51.583905 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6dfeeb6b16a4f92bc756bf8933c7fb1" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:51.583986 22372 raft_consensus.cc:2454] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:51.584061 22634 raft_consensus.cc:2454] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:51.584342 22701 leader_election.cc:302] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:51.584522 24337 raft_consensus.cc:2781] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:51.584589 24337 raft_consensus.cc:482] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:51.584635 24337 raft_consensus.cc:3037] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:51.586236 24337 raft_consensus.cc:504] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:51.586611 24337 leader_election.cc:288] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:51.586948 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6dfeeb6b16a4f92bc756bf8933c7fb1" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:51.587081 22372 raft_consensus.cc:3037] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:51.587069 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6dfeeb6b16a4f92bc756bf8933c7fb1" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:51.587208 22634 raft_consensus.cc:3037] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:51.587929 22372 raft_consensus.cc:2454] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:51.587993 22634 raft_consensus.cc:2454] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:51.588253 22701 leader_election.cc:302] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:51.588441 24337 raft_consensus.cc:2781] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:51.588541 24337 raft_consensus.cc:686] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:58:51.588722 24337 consensus_queue.cc:237] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:51.590516 22224 catalog_manager.cc:5484] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:58:51.739696 23792 raft_consensus.cc:482] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:51.739825 23792 raft_consensus.cc:504] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.740221 23792 leader_election.cc:288] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:51.740546 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6c248c0e15401b9eceeb6a428193dc" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:51.740669 22372 raft_consensus.cc:2454] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:51.740654 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6c248c0e15401b9eceeb6a428193dc" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:51.740818 22763 raft_consensus.cc:2454] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:51.740952 22439 leader_election.cc:302] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:51.741111 23792 raft_consensus.cc:2781] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:51.741165 23792 raft_consensus.cc:482] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:51.741211 23792 raft_consensus.cc:3037] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:51.742036 23792 raft_consensus.cc:504] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.742357 23792 leader_election.cc:288] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:51.742678 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6c248c0e15401b9eceeb6a428193dc" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:51.742784 22372 raft_consensus.cc:3037] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:51.742833 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef6c248c0e15401b9eceeb6a428193dc" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:51.742964 22763 raft_consensus.cc:3037] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:51.743554 22372 raft_consensus.cc:2454] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:58:51.743796 22439 leader_election.cc:302] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:51.743933 23792 raft_consensus.cc:2781] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:51.743952 22763 raft_consensus.cc:2454] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:58:51.744009 23792 raft_consensus.cc:686] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:58:51.744140 23792 consensus_queue.cc:237] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.746210 22224 catalog_manager.cc:5484] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:58:51.760098 24337 raft_consensus.cc:482] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:51.760254 24337 raft_consensus.cc:504] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.760772 24337 leader_election.cc:288] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:51.761219 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b935ed00d34452fb6c9d528d5f67cc5" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:51.761219 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b935ed00d34452fb6c9d528d5f67cc5" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:51.761385 22372 raft_consensus.cc:2454] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:51.761385 22501 raft_consensus.cc:2454] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:51.761729 22698 leader_election.cc:302] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:51.761895 24337 raft_consensus.cc:2781] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:51.761965 24337 raft_consensus.cc:482] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:51.762012 24337 raft_consensus.cc:3037] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:51.763137 24337 raft_consensus.cc:504] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.763478 24337 leader_election.cc:288] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:51.763775 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b935ed00d34452fb6c9d528d5f67cc5" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:51.763833 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b935ed00d34452fb6c9d528d5f67cc5" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:51.763900 22372 raft_consensus.cc:3037] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:51.763944 22501 raft_consensus.cc:3037] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:51.764674 22372 raft_consensus.cc:2454] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:51.764786 22501 raft_consensus.cc:2454] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:51.764919 22701 leader_election.cc:302] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:51.765095 24337 raft_consensus.cc:2781] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:51.765192 24337 raft_consensus.cc:686] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:58:51.765358 24337 consensus_queue.cc:237] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:51.767294 22224 catalog_manager.cc:5484] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
W20240502 07:58:51.889446 24342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:51.889683 24342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:51.893678 24342 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_14'
I20240502 07:58:51.990442 22763 raft_consensus.cc:1261] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:51.990901 22372 raft_consensus.cc:1261] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:51.990908 23792 consensus_queue.cc:1035] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:51.991295 23792 consensus_queue.cc:1035] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:51.994194 22634 raft_consensus.cc:1261] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:51.994591 24337 consensus_queue.cc:1035] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:51.997104 22370 raft_consensus.cc:1261] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:51.997524 23851 consensus_queue.cc:1035] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:51.999187 22371 raft_consensus.cc:1261] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:51.999552 23851 consensus_queue.cc:1035] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:52.000500 22501 raft_consensus.cc:1261] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:52.001245 23851 consensus_queue.cc:1035] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:52.039464 24263 mvcc.cc:205] Tried to move back new op lower bound from 7023152054262702080 to 7023152053312479232. Current Snapshot: MvccSnapshot[applied={T|T < 7023152054262702080}]
INSERT report
rows total: 2000
time total: 124.167 ms
time per row: 0.0620835 ms
I20240502 07:58:52.064973 22223 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_15"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:52.070760 22612 tablet_service.cc:1467] Processing CreateTablet for tablet 170e3d20ee76481c9f709d388481a58a (DEFAULT_TABLE table=rebalance_test_table_15 [id=4e81cbd084694ac8b1db3f5574dde601]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:52.070902 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 170e3d20ee76481c9f709d388481a58a (DEFAULT_TABLE table=rebalance_test_table_15 [id=4e81cbd084694ac8b1db3f5574dde601]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:52.071117 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 170e3d20ee76481c9f709d388481a58a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:52.071177 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 170e3d20ee76481c9f709d388481a58a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:52.071732 22613 tablet_service.cc:1467] Processing CreateTablet for tablet c5a30e1ebd7747eb911fba8437e42077 (DEFAULT_TABLE table=rebalance_test_table_15 [id=4e81cbd084694ac8b1db3f5574dde601]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:52.071846 22350 tablet_service.cc:1467] Processing CreateTablet for tablet c5a30e1ebd7747eb911fba8437e42077 (DEFAULT_TABLE table=rebalance_test_table_15 [id=4e81cbd084694ac8b1db3f5574dde601]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:52.071930 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5a30e1ebd7747eb911fba8437e42077. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:52.072067 22350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5a30e1ebd7747eb911fba8437e42077. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:52.073978 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 170e3d20ee76481c9f709d388481a58a (DEFAULT_TABLE table=rebalance_test_table_15 [id=4e81cbd084694ac8b1db3f5574dde601]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:52.074249 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 170e3d20ee76481c9f709d388481a58a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:52.074352 22744 tablet_service.cc:1467] Processing CreateTablet for tablet c5a30e1ebd7747eb911fba8437e42077 (DEFAULT_TABLE table=rebalance_test_table_15 [id=4e81cbd084694ac8b1db3f5574dde601]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:52.074618 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5a30e1ebd7747eb911fba8437e42077. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:52.075006 22613 tablet_service.cc:1467] Processing CreateTablet for tablet 895f9a31785d4558acd3b64904a645ca (DEFAULT_TABLE table=rebalance_test_table_15 [id=4e81cbd084694ac8b1db3f5574dde601]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:52.075218 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 895f9a31785d4558acd3b64904a645ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:52.076088 24393 tablet_bootstrap.cc:492] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:52.076561 22482 tablet_service.cc:1467] Processing CreateTablet for tablet 895f9a31785d4558acd3b64904a645ca (DEFAULT_TABLE table=rebalance_test_table_15 [id=4e81cbd084694ac8b1db3f5574dde601]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:52.076786 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 895f9a31785d4558acd3b64904a645ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:52.076972 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 895f9a31785d4558acd3b64904a645ca (DEFAULT_TABLE table=rebalance_test_table_15 [id=4e81cbd084694ac8b1db3f5574dde601]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:52.077035 24395 tablet_bootstrap.cc:492] T 170e3d20ee76481c9f709d388481a58a P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:52.077198 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 895f9a31785d4558acd3b64904a645ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:52.077507 24393 tablet_bootstrap.cc:654] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:52.078117 24395 tablet_bootstrap.cc:654] T 170e3d20ee76481c9f709d388481a58a P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:52.078528 24393 tablet_bootstrap.cc:492] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:52.078603 24393 ts_tablet_manager.cc:1367] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:52.079027 24393 raft_consensus.cc:348] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:52.079136 24393 raft_consensus.cc:374] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:52.079173 24393 raft_consensus.cc:729] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:52.079308 24393 consensus_queue.cc:260] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:52.079758 24396 tablet_bootstrap.cc:492] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:52.080695 24393 ts_tablet_manager.cc:1398] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:52.080735 24396 tablet_bootstrap.cc:654] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:52.080983 24393 tablet_bootstrap.cc:492] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:52.081876 24393 tablet_bootstrap.cc:654] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:52.081791 24394 tablet_bootstrap.cc:492] T 170e3d20ee76481c9f709d388481a58a P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:52.082798 24396 tablet_bootstrap.cc:492] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:52.082867 24396 ts_tablet_manager.cc:1367] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:52.083632 24394 tablet_bootstrap.cc:654] T 170e3d20ee76481c9f709d388481a58a P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:52.083258 24396 raft_consensus.cc:348] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:52.084132 24396 raft_consensus.cc:374] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:52.084172 24396 raft_consensus.cc:729] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:52.084869 24394 tablet_bootstrap.cc:492] T 170e3d20ee76481c9f709d388481a58a P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:52.085021 24394 ts_tablet_manager.cc:1367] T 170e3d20ee76481c9f709d388481a58a P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:52.085534 24394 raft_consensus.cc:348] T 170e3d20ee76481c9f709d388481a58a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:52.085650 24394 raft_consensus.cc:374] T 170e3d20ee76481c9f709d388481a58a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:52.085687 24394 raft_consensus.cc:729] T 170e3d20ee76481c9f709d388481a58a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:52.085718 24395 tablet_bootstrap.cc:492] T 170e3d20ee76481c9f709d388481a58a P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:52.085810 24395 ts_tablet_manager.cc:1367] T 170e3d20ee76481c9f709d388481a58a P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 07:58:52.085863 24394 consensus_queue.cc:260] T 170e3d20ee76481c9f709d388481a58a P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:52.086212 24395 raft_consensus.cc:348] T 170e3d20ee76481c9f709d388481a58a P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:52.086310 24395 raft_consensus.cc:374] T 170e3d20ee76481c9f709d388481a58a P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:52.086303 24394 ts_tablet_manager.cc:1398] T 170e3d20ee76481c9f709d388481a58a P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:52.086339 24395 raft_consensus.cc:729] T 170e3d20ee76481c9f709d388481a58a P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:52.086469 24394 tablet_bootstrap.cc:492] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:52.086441 24395 consensus_queue.cc:260] T 170e3d20ee76481c9f709d388481a58a P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:52.086673 24395 ts_tablet_manager.cc:1398] T 170e3d20ee76481c9f709d388481a58a P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:52.086820 24395 tablet_bootstrap.cc:492] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:52.087731 24395 tablet_bootstrap.cc:654] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:52.088382 24394 tablet_bootstrap.cc:654] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:52.085050 24396 consensus_queue.cc:260] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:52.090130 24396 ts_tablet_manager.cc:1398] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.007s user 0.000s sys 0.001s
I20240502 07:58:52.090327 24396 tablet_bootstrap.cc:492] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:52.091202 24396 tablet_bootstrap.cc:654] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:52.093755 24396 tablet_bootstrap.cc:492] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:52.093842 24396 ts_tablet_manager.cc:1367] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 07:58:52.093943 24393 tablet_bootstrap.cc:492] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:52.093941 24395 tablet_bootstrap.cc:492] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:52.094033 24395 ts_tablet_manager.cc:1367] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 07:58:52.094033 24393 ts_tablet_manager.cc:1367] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20240502 07:58:52.094264 24396 raft_consensus.cc:348] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:52.094362 24396 raft_consensus.cc:374] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:52.094403 24396 raft_consensus.cc:729] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:52.094434 24393 raft_consensus.cc:348] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:52.094440 24395 raft_consensus.cc:348] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:52.094507 24396 consensus_queue.cc:260] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:52.094532 24393 raft_consensus.cc:374] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:52.094532 24395 raft_consensus.cc:374] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:52.094564 24395 raft_consensus.cc:729] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:52.094564 24393 raft_consensus.cc:729] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:52.094684 24395 consensus_queue.cc:260] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:52.094681 24393 consensus_queue.cc:260] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:52.094738 24396 ts_tablet_manager.cc:1398] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:52.094944 24395 ts_tablet_manager.cc:1398] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:52.094974 24393 ts_tablet_manager.cc:1398] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:52.095130 24393 tablet_bootstrap.cc:492] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:52.095942 24394 tablet_bootstrap.cc:492] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:52.095971 24393 tablet_bootstrap.cc:654] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:52.096025 24394 ts_tablet_manager.cc:1367] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240502 07:58:52.096525 24394 raft_consensus.cc:348] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:52.096648 24394 raft_consensus.cc:374] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:52.096701 24394 raft_consensus.cc:729] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:52.096820 24394 consensus_queue.cc:260] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:52.098512 24394 ts_tablet_manager.cc:1398] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:52.099520 24393 tablet_bootstrap.cc:492] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:52.099592 24393 ts_tablet_manager.cc:1367] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:52.099992 24393 raft_consensus.cc:348] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:52.100077 24393 raft_consensus.cc:374] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:52.100104 24393 raft_consensus.cc:729] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:52.100211 24393 consensus_queue.cc:260] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:52.100432 24393 ts_tablet_manager.cc:1398] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:52.128362 23851 raft_consensus.cc:482] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:52.128494 23851 raft_consensus.cc:504] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:52.128886 23851 leader_election.cc:288] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:52.129228 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5a30e1ebd7747eb911fba8437e42077" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:52.129346 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5a30e1ebd7747eb911fba8437e42077" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:52.129385 22634 raft_consensus.cc:2454] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:52.129477 22371 raft_consensus.cc:2454] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:52.129714 22701 leader_election.cc:302] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:52.129881 24337 raft_consensus.cc:2781] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:52.129938 24337 raft_consensus.cc:482] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:52.129992 24337 raft_consensus.cc:3037] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:52.130769 24337 raft_consensus.cc:504] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:52.131078 24337 leader_election.cc:288] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:52.131372 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5a30e1ebd7747eb911fba8437e42077" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:52.131414 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5a30e1ebd7747eb911fba8437e42077" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:52.131518 22371 raft_consensus.cc:3037] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:52.131644 22634 raft_consensus.cc:3037] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:52.132294 22371 raft_consensus.cc:2454] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:52.132440 22634 raft_consensus.cc:2454] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:52.132588 22701 leader_election.cc:302] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:52.132783 24337 raft_consensus.cc:2781] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:52.132867 24337 raft_consensus.cc:686] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:58:52.133039 24337 consensus_queue.cc:237] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:52.135020 22223 catalog_manager.cc:5484] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:58:52.275897 24276 raft_consensus.cc:482] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:52.276013 24276 raft_consensus.cc:504] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:52.276481 24276 leader_election.cc:288] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:52.276850 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "170e3d20ee76481c9f709d388481a58a" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:52.276897 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "170e3d20ee76481c9f709d388481a58a" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:52.277017 22371 raft_consensus.cc:2454] T 170e3d20ee76481c9f709d388481a58a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:52.277030 22501 raft_consensus.cc:2454] T 170e3d20ee76481c9f709d388481a58a P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:52.277313 22570 leader_election.cc:302] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:52.277693 24276 raft_consensus.cc:2781] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:52.277752 24276 raft_consensus.cc:482] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:52.277806 24276 raft_consensus.cc:3037] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:52.278692 24276 raft_consensus.cc:504] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:52.279013 24276 leader_election.cc:288] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:52.279326 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "170e3d20ee76481c9f709d388481a58a" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:52.279332 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "170e3d20ee76481c9f709d388481a58a" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:52.279418 22501 raft_consensus.cc:3037] T 170e3d20ee76481c9f709d388481a58a P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:52.279428 22371 raft_consensus.cc:3037] T 170e3d20ee76481c9f709d388481a58a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:52.280179 22501 raft_consensus.cc:2454] T 170e3d20ee76481c9f709d388481a58a P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:52.280194 22371 raft_consensus.cc:2454] T 170e3d20ee76481c9f709d388481a58a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:52.280459 22567 leader_election.cc:302] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:52.280619 24276 raft_consensus.cc:2781] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:52.280718 24276 raft_consensus.cc:686] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:52.280882 24276 consensus_queue.cc:237] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:52.283028 22223 catalog_manager.cc:5484] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:58:52.283800 24337 raft_consensus.cc:482] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:52.283929 24337 raft_consensus.cc:504] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:52.284289 24337 leader_election.cc:288] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:52.284637 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "895f9a31785d4558acd3b64904a645ca" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:52.284725 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "895f9a31785d4558acd3b64904a645ca" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:52.284747 22634 raft_consensus.cc:2454] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:52.284816 22501 raft_consensus.cc:2454] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:52.285135 22698 leader_election.cc:302] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:52.285367 24337 raft_consensus.cc:2781] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:52.285436 24337 raft_consensus.cc:482] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:52.285488 24337 raft_consensus.cc:3037] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:52.286330 24337 raft_consensus.cc:504] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:52.286682 24337 leader_election.cc:288] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:52.286974 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "895f9a31785d4558acd3b64904a645ca" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:52.287082 22503 raft_consensus.cc:3037] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:52.287186 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "895f9a31785d4558acd3b64904a645ca" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:52.287284 22634 raft_consensus.cc:3037] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:52.288110 22503 raft_consensus.cc:2454] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:52.288190 22634 raft_consensus.cc:2454] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:52.288447 22701 leader_election.cc:302] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:52.288599 24337 raft_consensus.cc:2781] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:52.288697 24337 raft_consensus.cc:686] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:58:52.288837 24337 consensus_queue.cc:237] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:52.290527 22223 catalog_manager.cc:5484] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
W20240502 07:58:52.427795 24409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:52.428035 24409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:52.431124 24409 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_15'
I20240502 07:58:52.533788 22633 raft_consensus.cc:1261] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:52.534322 24337 consensus_queue.cc:1035] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:52.535018 22503 raft_consensus.cc:1261] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:52.535486 23851 consensus_queue.cc:1035] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:52.535624 22503 raft_consensus.cc:1261] T 170e3d20ee76481c9f709d388481a58a P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:52.536161 24276 consensus_queue.cc:1035] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:52.548496 22371 raft_consensus.cc:1261] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:52.549801 23851 consensus_queue.cc:1035] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:52.550067 24442 mvcc.cc:205] Tried to move back new op lower bound from 7023152056453799936 to 7023152055456223232. Current Snapshot: MvccSnapshot[applied={T|T < 7023152056453799936}]
I20240502 07:58:52.551771 22633 raft_consensus.cc:1261] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:52.557566 24453 consensus_queue.cc:1035] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:52.573338 22371 raft_consensus.cc:1261] T 170e3d20ee76481c9f709d388481a58a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:52.573798 24459 consensus_queue.cc:1035] T 170e3d20ee76481c9f709d388481a58a P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 136.007 ms
time per row: 0.0680036 ms
I20240502 07:58:52.610955 22222 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_16"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:52.617285 22350 tablet_service.cc:1467] Processing CreateTablet for tablet ca6b1845b2f8405da3ffa3fcb9d86150 (DEFAULT_TABLE table=rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:52.617650 22350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca6b1845b2f8405da3ffa3fcb9d86150. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:52.618297 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 9dddf3cd3f104977b369e787913af039 (DEFAULT_TABLE table=rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:52.618337 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 9dddf3cd3f104977b369e787913af039 (DEFAULT_TABLE table=rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:52.618361 22744 tablet_service.cc:1467] Processing CreateTablet for tablet fa14f8f327cb4221aabeedac16d258b6 (DEFAULT_TABLE table=rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:52.618554 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dddf3cd3f104977b369e787913af039. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:52.618552 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dddf3cd3f104977b369e787913af039. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:52.618676 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa14f8f327cb4221aabeedac16d258b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:52.620158 22351 tablet_service.cc:1467] Processing CreateTablet for tablet fa14f8f327cb4221aabeedac16d258b6 (DEFAULT_TABLE table=rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:52.620419 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa14f8f327cb4221aabeedac16d258b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:52.620988 24467 tablet_bootstrap.cc:492] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:52.621152 22612 tablet_service.cc:1467] Processing CreateTablet for tablet fa14f8f327cb4221aabeedac16d258b6 (DEFAULT_TABLE table=rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:52.621460 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa14f8f327cb4221aabeedac16d258b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:52.621670 22613 tablet_service.cc:1467] Processing CreateTablet for tablet ca6b1845b2f8405da3ffa3fcb9d86150 (DEFAULT_TABLE table=rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:52.621891 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca6b1845b2f8405da3ffa3fcb9d86150. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:52.622002 24467 tablet_bootstrap.cc:654] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:52.623118 24467 tablet_bootstrap.cc:492] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:52.623189 24467 ts_tablet_manager.cc:1367] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:52.623414 24470 tablet_bootstrap.cc:492] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:52.623942 24467 raft_consensus.cc:348] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:52.624053 24467 raft_consensus.cc:374] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:52.624092 24467 raft_consensus.cc:729] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:52.624233 24467 consensus_queue.cc:260] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:52.624302 24470 tablet_bootstrap.cc:654] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:52.624359 22482 tablet_service.cc:1467] Processing CreateTablet for tablet ca6b1845b2f8405da3ffa3fcb9d86150 (DEFAULT_TABLE table=rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:52.624365 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 9dddf3cd3f104977b369e787913af039 (DEFAULT_TABLE table=rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:52.624511 24467 ts_tablet_manager.cc:1398] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:52.624644 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca6b1845b2f8405da3ffa3fcb9d86150. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:52.624816 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dddf3cd3f104977b369e787913af039. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:52.625276 24467 tablet_bootstrap.cc:492] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:52.625960 24470 tablet_bootstrap.cc:492] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:52.626053 24470 ts_tablet_manager.cc:1367] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:52.626216 24467 tablet_bootstrap.cc:654] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:52.626487 24470 raft_consensus.cc:348] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:52.626608 24470 raft_consensus.cc:374] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:52.626648 24470 raft_consensus.cc:729] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:52.626765 24470 consensus_queue.cc:260] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:52.627058 24470 ts_tablet_manager.cc:1398] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:52.627233 24470 tablet_bootstrap.cc:492] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:52.628077 24470 tablet_bootstrap.cc:654] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:52.628199 24473 tablet_bootstrap.cc:492] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:52.629043 24473 tablet_bootstrap.cc:654] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:52.629688 24470 tablet_bootstrap.cc:492] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:52.629717 24468 tablet_bootstrap.cc:492] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:52.629770 24470 ts_tablet_manager.cc:1367] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:52.630172 24470 raft_consensus.cc:348] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:52.630263 24470 raft_consensus.cc:374] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:52.630297 24470 raft_consensus.cc:729] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:52.630558 24470 consensus_queue.cc:260] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:52.631335 24467 tablet_bootstrap.cc:492] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:52.631464 24467 ts_tablet_manager.cc:1367] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240502 07:58:52.630947 24470 ts_tablet_manager.cc:1398] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:52.631592 24468 tablet_bootstrap.cc:654] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:52.631942 24467 raft_consensus.cc:348] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:52.632055 24467 raft_consensus.cc:374] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:52.632089 24467 raft_consensus.cc:729] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:52.632290 24467 consensus_queue.cc:260] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:52.632578 24467 ts_tablet_manager.cc:1398] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:52.633378 24467 tablet_bootstrap.cc:492] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:52.633757 24473 tablet_bootstrap.cc:492] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:52.633859 24473 ts_tablet_manager.cc:1367] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240502 07:58:52.634392 24473 raft_consensus.cc:348] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:52.634513 24473 raft_consensus.cc:374] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:52.634544 24473 raft_consensus.cc:729] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:52.634716 24473 consensus_queue.cc:260] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:52.635054 24473 ts_tablet_manager.cc:1398] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:52.635329 24473 tablet_bootstrap.cc:492] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:52.635437 24467 tablet_bootstrap.cc:654] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:52.635514 24468 tablet_bootstrap.cc:492] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:52.635658 24468 ts_tablet_manager.cc:1367] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:58:52.636175 24468 raft_consensus.cc:348] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:52.636353 24473 tablet_bootstrap.cc:654] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:52.636354 24468 raft_consensus.cc:374] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:52.636396 24468 raft_consensus.cc:729] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:52.636544 24467 tablet_bootstrap.cc:492] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:52.636533 24468 consensus_queue.cc:260] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:52.636615 24467 ts_tablet_manager.cc:1367] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:52.636904 24468 ts_tablet_manager.cc:1398] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:52.637090 24468 tablet_bootstrap.cc:492] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:52.637049 24467 raft_consensus.cc:348] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:52.637140 24467 raft_consensus.cc:374] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:52.637169 24467 raft_consensus.cc:729] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:52.637269 24467 consensus_queue.cc:260] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:52.637559 24467 ts_tablet_manager.cc:1398] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:52.637975 24468 tablet_bootstrap.cc:654] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:52.638170 24473 tablet_bootstrap.cc:492] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:52.638247 24473 ts_tablet_manager.cc:1367] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 07:58:52.638681 24473 raft_consensus.cc:348] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:52.638779 24473 raft_consensus.cc:374] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:52.638810 24473 raft_consensus.cc:729] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:52.638912 24468 tablet_bootstrap.cc:492] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:52.638967 24468 ts_tablet_manager.cc:1367] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:52.639240 24468 raft_consensus.cc:348] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:52.639307 24468 raft_consensus.cc:374] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:52.639325 24468 raft_consensus.cc:729] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:52.639323 24473 consensus_queue.cc:260] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:52.639418 24468 consensus_queue.cc:260] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:52.639644 24468 ts_tablet_manager.cc:1398] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:52.639640 24473 ts_tablet_manager.cc:1398] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 07:58:52.641613 24199 raft_consensus.cc:482] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:52.641708 24199 raft_consensus.cc:504] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:52.642055 24199 leader_election.cc:288] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:52.642470 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca6b1845b2f8405da3ffa3fcb9d86150" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:52.642479 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca6b1845b2f8405da3ffa3fcb9d86150" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:52.642580 22503 raft_consensus.cc:2454] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:52.642604 22634 raft_consensus.cc:2454] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:52.642930 22305 leader_election.cc:302] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:52.643105 24199 raft_consensus.cc:2781] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:52.643147 24199 raft_consensus.cc:482] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:52.643204 24199 raft_consensus.cc:3037] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:52.644095 24199 raft_consensus.cc:504] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:52.644398 24199 leader_election.cc:288] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:52.644678 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca6b1845b2f8405da3ffa3fcb9d86150" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:52.644716 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca6b1845b2f8405da3ffa3fcb9d86150" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:52.644776 22503 raft_consensus.cc:3037] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:52.644802 22634 raft_consensus.cc:3037] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:52.645589 22634 raft_consensus.cc:2454] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:52.645668 22503 raft_consensus.cc:2454] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:52.645890 22308 leader_election.cc:302] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:52.646059 24199 raft_consensus.cc:2781] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:52.646154 24199 raft_consensus.cc:686] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:58:52.646294 24199 consensus_queue.cc:237] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:52.647994 22222 catalog_manager.cc:5484] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:58:52.665735 24452 raft_consensus.cc:482] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:52.665886 24452 raft_consensus.cc:504] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:52.666393 24452 leader_election.cc:288] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:52.666652 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dddf3cd3f104977b369e787913af039" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:52.666805 22503 raft_consensus.cc:2454] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:52.666937 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dddf3cd3f104977b369e787913af039" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:52.667065 22698 leader_election.cc:302] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:52.667089 22371 raft_consensus.cc:2454] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:52.667214 24452 raft_consensus.cc:2781] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:52.667268 24452 raft_consensus.cc:482] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:52.667311 24452 raft_consensus.cc:3037] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:52.668315 24452 raft_consensus.cc:504] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:52.668614 24452 leader_election.cc:288] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:52.668880 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dddf3cd3f104977b369e787913af039" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:52.668987 22503 raft_consensus.cc:3037] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:52.668989 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dddf3cd3f104977b369e787913af039" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:52.669104 22371 raft_consensus.cc:3037] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:52.669811 22503 raft_consensus.cc:2454] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:52.669952 22371 raft_consensus.cc:2454] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:52.670025 22698 leader_election.cc:302] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:52.670184 24452 raft_consensus.cc:2781] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:52.670269 24452 raft_consensus.cc:686] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:58:52.670400 24452 consensus_queue.cc:237] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:52.672312 22222 catalog_manager.cc:5484] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:58:52.880951 24452 raft_consensus.cc:482] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:52.881094 24452 raft_consensus.cc:504] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:52.881628 24452 leader_election.cc:288] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:52.882131 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa14f8f327cb4221aabeedac16d258b6" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:52.882154 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa14f8f327cb4221aabeedac16d258b6" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:52.882295 22371 raft_consensus.cc:2454] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:52.882304 22634 raft_consensus.cc:2454] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:52.882642 22701 leader_election.cc:302] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:52.882848 24452 raft_consensus.cc:2781] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:52.882946 24452 raft_consensus.cc:482] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:52.882993 24452 raft_consensus.cc:3037] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:52.883924 24452 raft_consensus.cc:504] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:52.884243 24452 leader_election.cc:288] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:52.884562 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa14f8f327cb4221aabeedac16d258b6" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:52.884613 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa14f8f327cb4221aabeedac16d258b6" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:52.884672 22634 raft_consensus.cc:3037] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:52.884699 22371 raft_consensus.cc:3037] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:52.885416 22634 raft_consensus.cc:2454] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:52.885500 22371 raft_consensus.cc:2454] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:52.885723 22701 leader_election.cc:302] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:52.885905 24452 raft_consensus.cc:2781] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:52.885987 24452 raft_consensus.cc:686] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:58:52.886132 24452 consensus_queue.cc:237] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:52.888124 22222 catalog_manager.cc:5484] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
W20240502 07:58:53.031049 24483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:53.031291 24483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:53.034641 24483 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_16'
I20240502 07:58:53.062672 24432 consensus_queue.cc:1035] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:53.066280 24199 consensus_queue.cc:1035] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:53.121356 22371 raft_consensus.cc:1261] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:53.121786 24452 consensus_queue.cc:1035] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:53.127044 22634 raft_consensus.cc:1261] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:53.127444 24452 consensus_queue.cc:1035] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:53.130757 22370 raft_consensus.cc:1261] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:53.131274 22503 raft_consensus.cc:1261] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:53.131654 24452 consensus_queue.cc:1035] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:53.131873 24447 consensus_queue.cc:1035] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:53.142899 24514 mvcc.cc:205] Tried to move back new op lower bound from 7023152058853089280 to 7023152057902993408. Current Snapshot: MvccSnapshot[applied={T|T < 7023152058853089280}]
INSERT report
rows total: 2000
time total: 109.004 ms
time per row: 0.0545022 ms
I20240502 07:58:53.189811 22228 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:53.196066 22482 tablet_service.cc:1467] Processing CreateTablet for tablet b270d89546214f4d9d15193f4f8e04d5 (DEFAULT_TABLE table=rebalance_test_table_17 [id=1ec511fc66814cefa73a9b7acff4eea9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:53.196079 22744 tablet_service.cc:1467] Processing CreateTablet for tablet b270d89546214f4d9d15193f4f8e04d5 (DEFAULT_TABLE table=rebalance_test_table_17 [id=1ec511fc66814cefa73a9b7acff4eea9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:53.196367 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b270d89546214f4d9d15193f4f8e04d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:53.196383 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b270d89546214f4d9d15193f4f8e04d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:53.198089 22614 tablet_service.cc:1467] Processing CreateTablet for tablet 15f70916ad6248e2a24334020df2c55e (DEFAULT_TABLE table=rebalance_test_table_17 [id=1ec511fc66814cefa73a9b7acff4eea9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:53.198364 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15f70916ad6248e2a24334020df2c55e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:53.198416 22613 tablet_service.cc:1467] Processing CreateTablet for tablet b270d89546214f4d9d15193f4f8e04d5 (DEFAULT_TABLE table=rebalance_test_table_17 [id=1ec511fc66814cefa73a9b7acff4eea9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:53.198601 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b270d89546214f4d9d15193f4f8e04d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:53.198915 22744 tablet_service.cc:1467] Processing CreateTablet for tablet 3769936ffa2946fa9333ee78b8b41e76 (DEFAULT_TABLE table=rebalance_test_table_17 [id=1ec511fc66814cefa73a9b7acff4eea9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:53.199152 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3769936ffa2946fa9333ee78b8b41e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:53.199213 24539 tablet_bootstrap.cc:492] T b270d89546214f4d9d15193f4f8e04d5 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:53.198089 22612 tablet_service.cc:1467] Processing CreateTablet for tablet 3769936ffa2946fa9333ee78b8b41e76 (DEFAULT_TABLE table=rebalance_test_table_17 [id=1ec511fc66814cefa73a9b7acff4eea9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:53.199715 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3769936ffa2946fa9333ee78b8b41e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:53.200151 24539 tablet_bootstrap.cc:654] T b270d89546214f4d9d15193f4f8e04d5 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:53.200731 24538 tablet_bootstrap.cc:492] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:53.200845 22351 tablet_service.cc:1467] Processing CreateTablet for tablet 15f70916ad6248e2a24334020df2c55e (DEFAULT_TABLE table=rebalance_test_table_17 [id=1ec511fc66814cefa73a9b7acff4eea9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:53.201063 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15f70916ad6248e2a24334020df2c55e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:53.201648 24538 tablet_bootstrap.cc:654] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:53.202544 22482 tablet_service.cc:1467] Processing CreateTablet for tablet 3769936ffa2946fa9333ee78b8b41e76 (DEFAULT_TABLE table=rebalance_test_table_17 [id=1ec511fc66814cefa73a9b7acff4eea9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:53.202764 24539 tablet_bootstrap.cc:492] T b270d89546214f4d9d15193f4f8e04d5 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:53.202785 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3769936ffa2946fa9333ee78b8b41e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:53.202862 24539 ts_tablet_manager.cc:1367] T b270d89546214f4d9d15193f4f8e04d5 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:53.203370 24539 raft_consensus.cc:348] T b270d89546214f4d9d15193f4f8e04d5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.203495 24539 raft_consensus.cc:374] T b270d89546214f4d9d15193f4f8e04d5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:53.203541 24539 raft_consensus.cc:729] T b270d89546214f4d9d15193f4f8e04d5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:53.203675 24539 consensus_queue.cc:260] T b270d89546214f4d9d15193f4f8e04d5 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.203694 24543 tablet_bootstrap.cc:492] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:53.203974 24539 ts_tablet_manager.cc:1398] T b270d89546214f4d9d15193f4f8e04d5 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:53.204154 24539 tablet_bootstrap.cc:492] T 3769936ffa2946fa9333ee78b8b41e76 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:53.204586 24543 tablet_bootstrap.cc:654] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:53.204711 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 15f70916ad6248e2a24334020df2c55e (DEFAULT_TABLE table=rebalance_test_table_17 [id=1ec511fc66814cefa73a9b7acff4eea9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:53.204918 24539 tablet_bootstrap.cc:654] T 3769936ffa2946fa9333ee78b8b41e76 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:53.204931 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15f70916ad6248e2a24334020df2c55e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:53.205821 24543 tablet_bootstrap.cc:492] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:53.205914 24543 ts_tablet_manager.cc:1367] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:53.206424 24543 raft_consensus.cc:348] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.207365 24539 tablet_bootstrap.cc:492] T 3769936ffa2946fa9333ee78b8b41e76 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:53.207415 24543 raft_consensus.cc:374] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:53.207501 24543 raft_consensus.cc:729] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:53.207516 24539 ts_tablet_manager.cc:1367] T 3769936ffa2946fa9333ee78b8b41e76 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:53.207695 24543 consensus_queue.cc:260] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.208073 24543 ts_tablet_manager.cc:1398] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:53.208577 24540 tablet_bootstrap.cc:492] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:53.208864 24539 raft_consensus.cc:348] T 3769936ffa2946fa9333ee78b8b41e76 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.209050 24539 raft_consensus.cc:374] T 3769936ffa2946fa9333ee78b8b41e76 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:53.209134 24539 raft_consensus.cc:729] T 3769936ffa2946fa9333ee78b8b41e76 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:53.209354 24539 consensus_queue.cc:260] T 3769936ffa2946fa9333ee78b8b41e76 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.209534 24540 tablet_bootstrap.cc:654] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:53.209764 24539 ts_tablet_manager.cc:1398] T 3769936ffa2946fa9333ee78b8b41e76 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:53.210883 24538 tablet_bootstrap.cc:492] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:53.210932 24540 tablet_bootstrap.cc:492] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:53.210984 24538 ts_tablet_manager.cc:1367] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 07:58:53.210997 24540 ts_tablet_manager.cc:1367] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:53.211432 24540 raft_consensus.cc:348] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.211453 24538 raft_consensus.cc:348] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.211536 24540 raft_consensus.cc:374] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:53.211548 24538 raft_consensus.cc:374] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:53.211571 24540 raft_consensus.cc:729] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:53.211581 24538 raft_consensus.cc:729] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:53.211702 24538 consensus_queue.cc:260] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.211702 24540 consensus_queue.cc:260] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.211964 24540 ts_tablet_manager.cc:1398] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:53.211973 24538 ts_tablet_manager.cc:1398] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:53.212105 24540 tablet_bootstrap.cc:492] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:53.212105 24538 tablet_bootstrap.cc:492] T 3769936ffa2946fa9333ee78b8b41e76 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:53.212996 24538 tablet_bootstrap.cc:654] T 3769936ffa2946fa9333ee78b8b41e76 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:53.212997 24540 tablet_bootstrap.cc:654] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:53.214197 24538 tablet_bootstrap.cc:492] T 3769936ffa2946fa9333ee78b8b41e76 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:53.214272 24538 ts_tablet_manager.cc:1367] T 3769936ffa2946fa9333ee78b8b41e76 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 07:58:53.214692 24538 raft_consensus.cc:348] T 3769936ffa2946fa9333ee78b8b41e76 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.214794 24538 raft_consensus.cc:374] T 3769936ffa2946fa9333ee78b8b41e76 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:53.214828 24538 raft_consensus.cc:729] T 3769936ffa2946fa9333ee78b8b41e76 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:53.214928 24538 consensus_queue.cc:260] T 3769936ffa2946fa9333ee78b8b41e76 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.215194 24538 ts_tablet_manager.cc:1398] T 3769936ffa2946fa9333ee78b8b41e76 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:53.215343 24538 tablet_bootstrap.cc:492] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:53.216415 24538 tablet_bootstrap.cc:654] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:53.217243 24540 tablet_bootstrap.cc:492] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:53.217412 24540 ts_tablet_manager.cc:1367] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 07:58:53.217888 24540 raft_consensus.cc:348] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.218012 24540 raft_consensus.cc:374] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:53.218070 24540 raft_consensus.cc:729] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:53.218200 24540 consensus_queue.cc:260] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.218842 24540 ts_tablet_manager.cc:1398] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:53.219033 24540 tablet_bootstrap.cc:492] T 15f70916ad6248e2a24334020df2c55e P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:53.219964 24540 tablet_bootstrap.cc:654] T 15f70916ad6248e2a24334020df2c55e P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:53.220994 24540 tablet_bootstrap.cc:492] T 15f70916ad6248e2a24334020df2c55e P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:53.221071 24540 ts_tablet_manager.cc:1367] T 15f70916ad6248e2a24334020df2c55e P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 07:58:53.221488 24540 raft_consensus.cc:348] T 15f70916ad6248e2a24334020df2c55e P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.221573 24540 raft_consensus.cc:374] T 15f70916ad6248e2a24334020df2c55e P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:53.221624 24540 raft_consensus.cc:729] T 15f70916ad6248e2a24334020df2c55e P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:53.221630 24538 tablet_bootstrap.cc:492] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:53.221709 24538 ts_tablet_manager.cc:1367] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240502 07:58:53.221741 24540 consensus_queue.cc:260] T 15f70916ad6248e2a24334020df2c55e P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.221995 24540 ts_tablet_manager.cc:1398] T 15f70916ad6248e2a24334020df2c55e P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:53.222108 24538 raft_consensus.cc:348] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.222198 24538 raft_consensus.cc:374] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:53.222218 24538 raft_consensus.cc:729] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:53.222330 24538 consensus_queue.cc:260] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.222572 24538 ts_tablet_manager.cc:1398] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:53.256505 24459 raft_consensus.cc:482] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:53.256628 24459 raft_consensus.cc:504] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.257066 24459 leader_election.cc:288] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:53.257508 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3769936ffa2946fa9333ee78b8b41e76" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:53.257508 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3769936ffa2946fa9333ee78b8b41e76" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:53.257656 22763 raft_consensus.cc:2454] T 3769936ffa2946fa9333ee78b8b41e76 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:53.257678 22503 raft_consensus.cc:2454] T 3769936ffa2946fa9333ee78b8b41e76 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:53.258006 22570 leader_election.cc:302] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:53.258162 24459 raft_consensus.cc:2781] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:53.258216 24459 raft_consensus.cc:482] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:53.258270 24459 raft_consensus.cc:3037] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:53.259114 24459 raft_consensus.cc:504] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.259411 24459 leader_election.cc:288] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:53.259737 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3769936ffa2946fa9333ee78b8b41e76" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:53.259747 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3769936ffa2946fa9333ee78b8b41e76" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:53.259841 22763 raft_consensus.cc:3037] T 3769936ffa2946fa9333ee78b8b41e76 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:53.259840 22503 raft_consensus.cc:3037] T 3769936ffa2946fa9333ee78b8b41e76 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:53.260591 22763 raft_consensus.cc:2454] T 3769936ffa2946fa9333ee78b8b41e76 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:53.260680 22503 raft_consensus.cc:2454] T 3769936ffa2946fa9333ee78b8b41e76 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:53.260864 22570 leader_election.cc:302] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:53.261057 24459 raft_consensus.cc:2781] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:53.261135 24459 raft_consensus.cc:686] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:53.261262 24459 consensus_queue.cc:237] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.263257 22222 catalog_manager.cc:5484] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:58:53.274207 24432 raft_consensus.cc:482] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:53.274348 24432 raft_consensus.cc:504] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.274801 24432 leader_election.cc:288] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:53.275231 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15f70916ad6248e2a24334020df2c55e" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:53.275375 22633 raft_consensus.cc:2454] T 15f70916ad6248e2a24334020df2c55e P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:53.275645 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15f70916ad6248e2a24334020df2c55e" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:53.275683 22308 leader_election.cc:302] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:53.275780 22503 raft_consensus.cc:2454] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:53.275924 24432 raft_consensus.cc:2781] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:53.275983 24432 raft_consensus.cc:482] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:53.276023 24432 raft_consensus.cc:3037] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:53.276877 24432 raft_consensus.cc:504] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.277230 24432 leader_election.cc:288] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:53.277483 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15f70916ad6248e2a24334020df2c55e" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:53.277529 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15f70916ad6248e2a24334020df2c55e" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:53.277609 22503 raft_consensus.cc:3037] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:53.277693 22633 raft_consensus.cc:3037] T 15f70916ad6248e2a24334020df2c55e P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:53.278514 22503 raft_consensus.cc:2454] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:53.278707 22633 raft_consensus.cc:2454] T 15f70916ad6248e2a24334020df2c55e P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:53.278802 22305 leader_election.cc:302] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:53.278957 24432 raft_consensus.cc:2781] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:53.279029 24432 raft_consensus.cc:686] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:58:53.279165 24432 consensus_queue.cc:237] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.281212 22222 catalog_manager.cc:5484] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:58:53.469185 24459 raft_consensus.cc:482] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:53.469302 24459 raft_consensus.cc:504] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.469725 24459 leader_election.cc:288] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:53.470088 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b270d89546214f4d9d15193f4f8e04d5" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:53.470213 22763 raft_consensus.cc:2454] T b270d89546214f4d9d15193f4f8e04d5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:53.470456 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b270d89546214f4d9d15193f4f8e04d5" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:53.470518 22570 leader_election.cc:302] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:53.470592 22503 raft_consensus.cc:2454] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:53.470759 24459 raft_consensus.cc:2781] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:53.470817 24459 raft_consensus.cc:482] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:53.470844 24459 raft_consensus.cc:3037] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:53.471639 24459 raft_consensus.cc:504] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.471968 24459 leader_election.cc:288] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:53.472249 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b270d89546214f4d9d15193f4f8e04d5" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:53.472347 22503 raft_consensus.cc:3037] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:53.472306 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b270d89546214f4d9d15193f4f8e04d5" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:53.472445 22763 raft_consensus.cc:3037] T b270d89546214f4d9d15193f4f8e04d5 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:53.473075 22503 raft_consensus.cc:2454] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:53.473415 22763 raft_consensus.cc:2454] T b270d89546214f4d9d15193f4f8e04d5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:53.473726 22567 leader_election.cc:302] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:53.473963 24459 raft_consensus.cc:2781] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:53.474036 24459 raft_consensus.cc:686] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:53.474177 24459 consensus_queue.cc:237] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.476369 22222 catalog_manager.cc:5484] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
W20240502 07:58:53.632477 24554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:53.632777 24554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:53.635953 24554 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_17'
I20240502 07:58:53.721663 22763 raft_consensus.cc:1261] T b270d89546214f4d9d15193f4f8e04d5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:53.723129 24462 consensus_queue.cc:1035] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:53.723858 22503 raft_consensus.cc:1261] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:53.724097 22633 raft_consensus.cc:1261] T 15f70916ad6248e2a24334020df2c55e P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:53.724261 24199 consensus_queue.cc:1035] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:53.724486 24432 consensus_queue.cc:1035] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:53.732463 22765 raft_consensus.cc:1261] T 3769936ffa2946fa9333ee78b8b41e76 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:53.732698 22501 raft_consensus.cc:1261] T 3769936ffa2946fa9333ee78b8b41e76 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:53.733404 24459 consensus_queue.cc:1035] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:53.734306 24459 consensus_queue.cc:1035] T 3769936ffa2946fa9333ee78b8b41e76 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:53.735252 22502 raft_consensus.cc:1261] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:53.735605 24459 consensus_queue.cc:1035] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 123.906 ms
time per row: 0.0619531 ms
I20240502 07:58:53.803988 22222 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:53.809566 22351 tablet_service.cc:1467] Processing CreateTablet for tablet 4dc697950c8140878b50550fe0ff458d (DEFAULT_TABLE table=rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:53.809736 22352 tablet_service.cc:1467] Processing CreateTablet for tablet e5c1730dba9645ac90c5bb10def836a2 (DEFAULT_TABLE table=rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:53.809713 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 4dc697950c8140878b50550fe0ff458d (DEFAULT_TABLE table=rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:53.809878 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dc697950c8140878b50550fe0ff458d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:53.809983 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dc697950c8140878b50550fe0ff458d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:53.810062 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5c1730dba9645ac90c5bb10def836a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:53.811944 24609 tablet_bootstrap.cc:492] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:53.812225 22744 tablet_service.cc:1467] Processing CreateTablet for tablet 4dc697950c8140878b50550fe0ff458d (DEFAULT_TABLE table=rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:53.812474 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dc697950c8140878b50550fe0ff458d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:53.812925 24609 tablet_bootstrap.cc:654] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:53.814013 24609 tablet_bootstrap.cc:492] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:53.814108 24609 ts_tablet_manager.cc:1367] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:53.814625 24609 raft_consensus.cc:348] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:53.814826 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 132ac92527d24e18b7e4554cf47bbc01 (DEFAULT_TABLE table=rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:53.814837 22614 tablet_service.cc:1467] Processing CreateTablet for tablet e5c1730dba9645ac90c5bb10def836a2 (DEFAULT_TABLE table=rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:53.814826 24609 raft_consensus.cc:374] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:53.814960 24609 raft_consensus.cc:729] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:53.815068 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5c1730dba9645ac90c5bb10def836a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:53.815063 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 132ac92527d24e18b7e4554cf47bbc01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:53.815109 24609 consensus_queue.cc:260] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:53.815595 24610 tablet_bootstrap.cc:492] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:53.816399 24610 tablet_bootstrap.cc:654] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:53.815405 24609 ts_tablet_manager.cc:1398] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:53.816884 24609 tablet_bootstrap.cc:492] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:53.817003 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 132ac92527d24e18b7e4554cf47bbc01 (DEFAULT_TABLE table=rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:53.817240 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 132ac92527d24e18b7e4554cf47bbc01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:53.817371 24613 tablet_bootstrap.cc:492] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:53.817950 24609 tablet_bootstrap.cc:654] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:53.818204 24613 tablet_bootstrap.cc:654] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:53.819128 24609 tablet_bootstrap.cc:492] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:53.819217 24609 ts_tablet_manager.cc:1367] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:53.819401 24613 tablet_bootstrap.cc:492] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:53.819490 24613 ts_tablet_manager.cc:1367] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:53.819670 24609 raft_consensus.cc:348] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.819737 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 132ac92527d24e18b7e4554cf47bbc01 (DEFAULT_TABLE table=rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:53.819778 24609 raft_consensus.cc:374] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:53.819821 24609 raft_consensus.cc:729] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:53.819972 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 132ac92527d24e18b7e4554cf47bbc01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:53.819928 24613 raft_consensus.cc:348] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.819926 24609 consensus_queue.cc:260] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.820022 24613 raft_consensus.cc:374] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:53.820052 24613 raft_consensus.cc:729] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:53.820158 24613 consensus_queue.cc:260] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.820272 24609 ts_tablet_manager.cc:1398] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:53.820418 24609 tablet_bootstrap.cc:492] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:53.820433 24613 ts_tablet_manager.cc:1398] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:53.820580 24610 tablet_bootstrap.cc:492] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:53.820664 24610 ts_tablet_manager.cc:1367] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:53.821156 24610 raft_consensus.cc:348] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:53.821256 24609 tablet_bootstrap.cc:654] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:53.821298 24610 raft_consensus.cc:374] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:53.821357 24610 raft_consensus.cc:729] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:53.821471 24610 consensus_queue.cc:260] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:53.822315 22744 tablet_service.cc:1467] Processing CreateTablet for tablet e5c1730dba9645ac90c5bb10def836a2 (DEFAULT_TABLE table=rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:53.822530 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5c1730dba9645ac90c5bb10def836a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:53.823606 24609 tablet_bootstrap.cc:492] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:53.823695 24609 ts_tablet_manager.cc:1367] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:53.824111 24609 raft_consensus.cc:348] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:53.824208 24609 raft_consensus.cc:374] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:53.824246 24609 raft_consensus.cc:729] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:53.824368 24609 consensus_queue.cc:260] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:53.824657 24609 ts_tablet_manager.cc:1398] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:53.825438 24612 tablet_bootstrap.cc:492] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:53.825572 24610 ts_tablet_manager.cc:1398] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:58:53.825768 24610 tablet_bootstrap.cc:492] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:53.826596 24612 tablet_bootstrap.cc:654] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:53.826640 24610 tablet_bootstrap.cc:654] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:53.827543 24612 tablet_bootstrap.cc:492] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:53.827625 24612 ts_tablet_manager.cc:1367] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:53.828075 24612 raft_consensus.cc:348] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:53.828238 24612 raft_consensus.cc:374] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:53.828298 24612 raft_consensus.cc:729] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:53.828433 24612 consensus_queue.cc:260] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:53.828686 24612 ts_tablet_manager.cc:1398] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:53.828851 24612 tablet_bootstrap.cc:492] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:53.829644 24612 tablet_bootstrap.cc:654] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:53.829864 24610 tablet_bootstrap.cc:492] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:53.829946 24610 ts_tablet_manager.cc:1367] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:53.830632 24612 tablet_bootstrap.cc:492] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:53.830694 24612 ts_tablet_manager.cc:1367] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:53.830983 24610 raft_consensus.cc:348] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:53.831094 24610 raft_consensus.cc:374] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:53.831065 24612 raft_consensus.cc:348] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:53.831153 24610 raft_consensus.cc:729] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:53.831208 24612 raft_consensus.cc:374] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:53.831271 24612 raft_consensus.cc:729] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:53.831276 24610 consensus_queue.cc:260] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:53.831393 24612 consensus_queue.cc:260] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:53.831565 24610 ts_tablet_manager.cc:1398] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:53.831688 24612 ts_tablet_manager.cc:1398] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:53.831912 24612 tablet_bootstrap.cc:492] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:53.832767 24612 tablet_bootstrap.cc:654] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:53.833825 24612 tablet_bootstrap.cc:492] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:53.833915 24612 ts_tablet_manager.cc:1367] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:53.834246 24612 raft_consensus.cc:348] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.834376 24612 raft_consensus.cc:374] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:53.834430 24612 raft_consensus.cc:729] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:53.834571 24612 consensus_queue.cc:260] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:53.834934 24612 ts_tablet_manager.cc:1398] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:54.041707 24199 raft_consensus.cc:482] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:54.041823 24199 raft_consensus.cc:504] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:54.042228 24199 leader_election.cc:288] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:54.042601 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dc697950c8140878b50550fe0ff458d" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:54.042601 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dc697950c8140878b50550fe0ff458d" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:54.042750 22763 raft_consensus.cc:2454] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:54.042750 22503 raft_consensus.cc:2454] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:54.043038 22305 leader_election.cc:302] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:54.043254 24199 raft_consensus.cc:2781] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:54.043315 24199 raft_consensus.cc:482] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:54.043376 24199 raft_consensus.cc:3037] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:54.044215 24199 raft_consensus.cc:504] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:54.044546 24199 leader_election.cc:288] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:54.044812 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dc697950c8140878b50550fe0ff458d" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:54.044929 22763 raft_consensus.cc:3037] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:54.044970 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dc697950c8140878b50550fe0ff458d" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:54.045078 22502 raft_consensus.cc:3037] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:54.046010 22763 raft_consensus.cc:2454] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:54.046010 22502 raft_consensus.cc:2454] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:54.046316 22305 leader_election.cc:302] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:54.046531 24199 raft_consensus.cc:2781] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:54.046614 24199 raft_consensus.cc:686] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:58:54.046763 24199 consensus_queue.cc:237] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:54.048733 22222 catalog_manager.cc:5484] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:58:54.063233 24462 raft_consensus.cc:482] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:54.063345 24462 raft_consensus.cc:504] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:54.063776 24462 leader_election.cc:288] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:54.064142 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5c1730dba9645ac90c5bb10def836a2" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:54.064287 22372 raft_consensus.cc:2454] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:54.064617 22570 leader_election.cc:302] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:54.064774 24459 raft_consensus.cc:2781] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:54.064759 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5c1730dba9645ac90c5bb10def836a2" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:54.064826 24459 raft_consensus.cc:482] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:54.064888 24459 raft_consensus.cc:3037] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:54.064888 22763 raft_consensus.cc:2454] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:54.065774 24459 raft_consensus.cc:504] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:54.066090 24459 leader_election.cc:288] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:54.066382 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5c1730dba9645ac90c5bb10def836a2" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:54.066478 22372 raft_consensus.cc:3037] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:54.066452 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5c1730dba9645ac90c5bb10def836a2" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:54.066612 22763 raft_consensus.cc:3037] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:54.067251 22372 raft_consensus.cc:2454] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:54.067420 22763 raft_consensus.cc:2454] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:54.067540 22570 leader_election.cc:302] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:54.067736 24459 raft_consensus.cc:2781] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:54.067854 24459 raft_consensus.cc:686] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:54.067999 24459 consensus_queue.cc:237] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:54.071375 22222 catalog_manager.cc:5484] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:58:54.129266 24624 raft_consensus.cc:482] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:54.129441 24624 raft_consensus.cc:504] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:54.129940 24624 leader_election.cc:288] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:54.130411 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "132ac92527d24e18b7e4554cf47bbc01" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:54.130434 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "132ac92527d24e18b7e4554cf47bbc01" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:54.130566 22502 raft_consensus.cc:2454] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:54.130587 22372 raft_consensus.cc:2454] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:58:54.130918 22698 leader_election.cc:302] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:54.131108 24624 raft_consensus.cc:2781] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:54.131170 24624 raft_consensus.cc:482] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:54.131198 24624 raft_consensus.cc:3037] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:54.131985 24624 raft_consensus.cc:504] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:54.132287 24624 leader_election.cc:288] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:54.132671 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "132ac92527d24e18b7e4554cf47bbc01" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:54.132668 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "132ac92527d24e18b7e4554cf47bbc01" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:54.132771 22502 raft_consensus.cc:3037] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:54.132803 22372 raft_consensus.cc:3037] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:54.133574 22502 raft_consensus.cc:2454] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:54.133661 22372 raft_consensus.cc:2454] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:58:54.133877 22698 leader_election.cc:302] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:54.134093 24624 raft_consensus.cc:2781] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:54.134174 24624 raft_consensus.cc:686] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:58:54.134307 24624 consensus_queue.cc:237] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:54.136314 22222 catalog_manager.cc:5484] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
W20240502 07:58:54.296094 24626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:54.296319 24626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:54.299670 24626 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_18'
I20240502 07:58:54.398567 22372 raft_consensus.cc:1261] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:54.399147 22763 raft_consensus.cc:1261] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:54.399562 24459 consensus_queue.cc:1035] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:54.399876 24459 consensus_queue.cc:1035] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:54.401376 22372 raft_consensus.cc:1261] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:54.402951 24452 consensus_queue.cc:1035] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:54.404649 22502 raft_consensus.cc:1261] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:54.405758 24452 consensus_queue.cc:1035] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:54.422008 22502 raft_consensus.cc:1261] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:54.422502 24199 consensus_queue.cc:1035] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:54.427954 22765 raft_consensus.cc:1261] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:54.428712 24432 consensus_queue.cc:1035] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:54.439488 24663 mvcc.cc:205] Tried to move back new op lower bound from 7023152064092340224 to 7023152063015448576. Current Snapshot: MvccSnapshot[applied={T|T < 7023152064092340224}]
INSERT report
rows total: 2000
time total: 127.438 ms
time per row: 0.0637191 ms
I20240502 07:58:54.476954 22229 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalance_test_table_19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:54.482775 22614 tablet_service.cc:1467] Processing CreateTablet for tablet 41b4d8a50f684c19988a4cf1ee40921c (DEFAULT_TABLE table=rebalance_test_table_19 [id=4e87626760fa4d8098e55dfa6d1c55a9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:54.482853 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 41b4d8a50f684c19988a4cf1ee40921c (DEFAULT_TABLE table=rebalance_test_table_19 [id=4e87626760fa4d8098e55dfa6d1c55a9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:54.483110 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b4d8a50f684c19988a4cf1ee40921c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:54.483139 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b4d8a50f684c19988a4cf1ee40921c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:54.485100 24683 tablet_bootstrap.cc:492] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:54.485196 24684 tablet_bootstrap.cc:492] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:54.485754 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 41b4d8a50f684c19988a4cf1ee40921c (DEFAULT_TABLE table=rebalance_test_table_19 [id=4e87626760fa4d8098e55dfa6d1c55a9]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:54.485791 22351 tablet_service.cc:1467] Processing CreateTablet for tablet c2422c666baa48078030c43d1c0938f9 (DEFAULT_TABLE table=rebalance_test_table_19 [id=4e87626760fa4d8098e55dfa6d1c55a9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:54.486032 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b4d8a50f684c19988a4cf1ee40921c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:54.486022 22614 tablet_service.cc:1467] Processing CreateTablet for tablet c2422c666baa48078030c43d1c0938f9 (DEFAULT_TABLE table=rebalance_test_table_19 [id=4e87626760fa4d8098e55dfa6d1c55a9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:54.486110 22612 tablet_service.cc:1467] Processing CreateTablet for tablet 847f3a2601d44898a14a0b35409981ad (DEFAULT_TABLE table=rebalance_test_table_19 [id=4e87626760fa4d8098e55dfa6d1c55a9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:54.486213 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2422c666baa48078030c43d1c0938f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:54.486321 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 847f3a2601d44898a14a0b35409981ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:54.486378 24683 tablet_bootstrap.cc:654] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:54.488435 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2422c666baa48078030c43d1c0938f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:54.488612 24686 tablet_bootstrap.cc:492] T 41b4d8a50f684c19988a4cf1ee40921c P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:54.488586 22744 tablet_service.cc:1467] Processing CreateTablet for tablet c2422c666baa48078030c43d1c0938f9 (DEFAULT_TABLE table=rebalance_test_table_19 [id=4e87626760fa4d8098e55dfa6d1c55a9]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:54.488837 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2422c666baa48078030c43d1c0938f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:54.489524 24686 tablet_bootstrap.cc:654] T 41b4d8a50f684c19988a4cf1ee40921c P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:54.490645 24686 tablet_bootstrap.cc:492] T 41b4d8a50f684c19988a4cf1ee40921c P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:54.490796 24686 ts_tablet_manager.cc:1367] T 41b4d8a50f684c19988a4cf1ee40921c P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:54.491178 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 847f3a2601d44898a14a0b35409981ad (DEFAULT_TABLE table=rebalance_test_table_19 [id=4e87626760fa4d8098e55dfa6d1c55a9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:54.491261 24686 raft_consensus.cc:348] T 41b4d8a50f684c19988a4cf1ee40921c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:54.491339 24684 tablet_bootstrap.cc:654] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:54.491387 24686 raft_consensus.cc:374] T 41b4d8a50f684c19988a4cf1ee40921c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:54.491432 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 847f3a2601d44898a14a0b35409981ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:54.491432 24686 raft_consensus.cc:729] T 41b4d8a50f684c19988a4cf1ee40921c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:54.491585 24686 consensus_queue.cc:260] T 41b4d8a50f684c19988a4cf1ee40921c P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:54.491855 24686 ts_tablet_manager.cc:1398] T 41b4d8a50f684c19988a4cf1ee40921c P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:54.492058 24686 tablet_bootstrap.cc:492] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:54.492480 24684 tablet_bootstrap.cc:492] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:54.492640 24684 ts_tablet_manager.cc:1367] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:58:54.492839 24686 tablet_bootstrap.cc:654] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:54.493489 24684 raft_consensus.cc:348] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:54.493605 24684 raft_consensus.cc:374] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:54.493638 24684 raft_consensus.cc:729] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:54.493803 24684 consensus_queue.cc:260] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:54.493911 24686 tablet_bootstrap.cc:492] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:54.494107 24686 ts_tablet_manager.cc:1367] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:54.494133 24684 ts_tablet_manager.cc:1398] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:54.494570 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 847f3a2601d44898a14a0b35409981ad (DEFAULT_TABLE table=rebalance_test_table_19 [id=4e87626760fa4d8098e55dfa6d1c55a9]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:54.494644 24688 tablet_bootstrap.cc:492] T c2422c666baa48078030c43d1c0938f9 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:54.494792 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 847f3a2601d44898a14a0b35409981ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:54.495548 24688 tablet_bootstrap.cc:654] T c2422c666baa48078030c43d1c0938f9 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:54.496440 24684 tablet_bootstrap.cc:492] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:54.496747 24683 tablet_bootstrap.cc:492] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:54.496834 24683 ts_tablet_manager.cc:1367] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 07:58:54.497259 24683 raft_consensus.cc:348] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:54.497527 24684 tablet_bootstrap.cc:654] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:54.497541 24683 raft_consensus.cc:374] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:54.497586 24683 raft_consensus.cc:729] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:54.497727 24683 consensus_queue.cc:260] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:54.497997 24688 tablet_bootstrap.cc:492] T c2422c666baa48078030c43d1c0938f9 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:54.498013 24683 ts_tablet_manager.cc:1398] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:54.498080 24688 ts_tablet_manager.cc:1367] T c2422c666baa48078030c43d1c0938f9 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:54.498178 24683 tablet_bootstrap.cc:492] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:54.498255 24686 raft_consensus.cc:348] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:54.498364 24686 raft_consensus.cc:374] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:54.498394 24686 raft_consensus.cc:729] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:54.498510 24686 consensus_queue.cc:260] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:54.498781 24686 ts_tablet_manager.cc:1398] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.005s user 0.000s sys 0.000s
I20240502 07:58:54.498708 24688 raft_consensus.cc:348] T c2422c666baa48078030c43d1c0938f9 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:54.498935 24688 raft_consensus.cc:374] T c2422c666baa48078030c43d1c0938f9 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:54.498996 24688 raft_consensus.cc:729] T c2422c666baa48078030c43d1c0938f9 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:54.499006 24683 tablet_bootstrap.cc:654] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:54.499128 24688 consensus_queue.cc:260] T c2422c666baa48078030c43d1c0938f9 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:54.499441 24688 ts_tablet_manager.cc:1398] T c2422c666baa48078030c43d1c0938f9 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:54.499619 24688 tablet_bootstrap.cc:492] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:54.500607 24688 tablet_bootstrap.cc:654] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:54.503132 24683 tablet_bootstrap.cc:492] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:54.503221 24683 ts_tablet_manager.cc:1367] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:58:54.504068 24683 raft_consensus.cc:348] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:54.504220 24683 raft_consensus.cc:374] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:54.504272 24683 raft_consensus.cc:729] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:54.504364 24683 consensus_queue.cc:260] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:54.504650 24683 ts_tablet_manager.cc:1398] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:54.504645 24684 tablet_bootstrap.cc:492] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:54.504774 24684 ts_tablet_manager.cc:1367] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:58:54.504817 24683 tablet_bootstrap.cc:492] T 847f3a2601d44898a14a0b35409981ad P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:54.505754 24683 tablet_bootstrap.cc:654] T 847f3a2601d44898a14a0b35409981ad P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:54.507148 24683 tablet_bootstrap.cc:492] T 847f3a2601d44898a14a0b35409981ad P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:54.507277 24683 ts_tablet_manager.cc:1367] T 847f3a2601d44898a14a0b35409981ad P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:54.507227 24684 raft_consensus.cc:348] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:54.507390 24684 raft_consensus.cc:374] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:54.507454 24684 raft_consensus.cc:729] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:54.507634 24684 consensus_queue.cc:260] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:54.507855 24683 raft_consensus.cc:348] T 847f3a2601d44898a14a0b35409981ad P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:54.507989 24684 ts_tablet_manager.cc:1398] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:54.507997 24683 raft_consensus.cc:374] T 847f3a2601d44898a14a0b35409981ad P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:54.508783 24688 tablet_bootstrap.cc:492] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:54.508647 24683 raft_consensus.cc:729] T 847f3a2601d44898a14a0b35409981ad P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:54.508924 24688 ts_tablet_manager.cc:1367] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 07:58:54.509011 24683 consensus_queue.cc:260] T 847f3a2601d44898a14a0b35409981ad P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:54.509406 24683 ts_tablet_manager.cc:1398] T 847f3a2601d44898a14a0b35409981ad P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:54.509610 24688 raft_consensus.cc:348] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:54.509768 24688 raft_consensus.cc:374] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:54.509842 24688 raft_consensus.cc:729] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:54.509995 24688 consensus_queue.cc:260] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:54.510815 24688 ts_tablet_manager.cc:1398] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:54.510927 24592 raft_consensus.cc:482] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:54.511021 24592 raft_consensus.cc:504] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:54.511474 24592 leader_election.cc:288] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:54.512262 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847f3a2601d44898a14a0b35409981ad" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:54.512387 22634 raft_consensus.cc:2454] T 847f3a2601d44898a14a0b35409981ad P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:54.512741 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847f3a2601d44898a14a0b35409981ad" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:54.512876 22763 raft_consensus.cc:2454] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:54.512861 22439 leader_election.cc:302] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:54.513021 24592 raft_consensus.cc:2781] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:54.513078 24592 raft_consensus.cc:482] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:54.513126 24592 raft_consensus.cc:3037] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:54.514206 24592 raft_consensus.cc:504] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:54.514523 24592 leader_election.cc:288] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:54.514963 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847f3a2601d44898a14a0b35409981ad" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:54.515067 22763 raft_consensus.cc:3037] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:54.515094 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847f3a2601d44898a14a0b35409981ad" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:54.515194 22634 raft_consensus.cc:3037] T 847f3a2601d44898a14a0b35409981ad P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:54.516104 22763 raft_consensus.cc:2454] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:58:54.516197 22634 raft_consensus.cc:2454] T 847f3a2601d44898a14a0b35409981ad P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:58:54.516407 22439 leader_election.cc:302] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:54.516578 24592 raft_consensus.cc:2781] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:54.516665 24592 raft_consensus.cc:686] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:58:54.516788 24592 consensus_queue.cc:237] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:58:54.518853 22229 catalog_manager.cc:5484] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:58:54.525816 24459 raft_consensus.cc:482] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:54.525940 24459 raft_consensus.cc:504] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:54.526350 24459 leader_election.cc:288] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:54.526695 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b4d8a50f684c19988a4cf1ee40921c" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:54.526710 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b4d8a50f684c19988a4cf1ee40921c" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:54.526832 22371 raft_consensus.cc:2454] T 41b4d8a50f684c19988a4cf1ee40921c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:54.526841 22502 raft_consensus.cc:2454] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:54.527156 22570 leader_election.cc:302] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:54.527401 24459 raft_consensus.cc:2781] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:54.527458 24459 raft_consensus.cc:482] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:54.527511 24459 raft_consensus.cc:3037] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:54.528155 24432 raft_consensus.cc:482] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:54.528264 24432 raft_consensus.cc:504] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:54.528471 24459 raft_consensus.cc:504] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:54.528786 24432 leader_election.cc:288] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:54.528839 24459 leader_election.cc:288] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:54.529165 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2422c666baa48078030c43d1c0938f9" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:54.529175 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2422c666baa48078030c43d1c0938f9" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:54.529294 22634 raft_consensus.cc:2454] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:54.529294 22763 raft_consensus.cc:2454] T c2422c666baa48078030c43d1c0938f9 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:58:54.529438 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b4d8a50f684c19988a4cf1ee40921c" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:54.529531 22371 raft_consensus.cc:3037] T 41b4d8a50f684c19988a4cf1ee40921c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:54.529642 22308 leader_election.cc:302] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:54.529898 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b4d8a50f684c19988a4cf1ee40921c" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:54.530017 22502 raft_consensus.cc:3037] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:54.530508 22371 raft_consensus.cc:2454] T 41b4d8a50f684c19988a4cf1ee40921c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:54.530795 22502 raft_consensus.cc:2454] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:54.530908 22570 leader_election.cc:302] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:54.531070 24459 raft_consensus.cc:2781] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:54.531147 24459 raft_consensus.cc:686] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:54.531277 24459 consensus_queue.cc:237] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:54.532881 24432 raft_consensus.cc:2781] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:54.532958 24432 raft_consensus.cc:482] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:54.533008 24432 raft_consensus.cc:3037] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:54.533756 22229 catalog_manager.cc:5484] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:58:54.534088 24432 raft_consensus.cc:504] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:54.534476 24432 leader_election.cc:288] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:54.534977 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2422c666baa48078030c43d1c0938f9" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:54.534977 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2422c666baa48078030c43d1c0938f9" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:54.535094 22634 raft_consensus.cc:3037] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:54.535095 22763 raft_consensus.cc:3037] T c2422c666baa48078030c43d1c0938f9 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:54.536041 22634 raft_consensus.cc:2454] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:54.536063 22763 raft_consensus.cc:2454] T c2422c666baa48078030c43d1c0938f9 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:58:54.536437 22308 leader_election.cc:302] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:54.536707 24432 raft_consensus.cc:2781] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:54.536814 24432 raft_consensus.cc:686] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:58:54.536959 24432 consensus_queue.cc:237] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:54.539172 22229 catalog_manager.cc:5484] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
W20240502 07:58:54.661952 24700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:54.662259 24700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:54.665527 24700 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_19'
I20240502 07:58:54.766078 22502 raft_consensus.cc:1261] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:54.766211 22372 raft_consensus.cc:1261] T 41b4d8a50f684c19988a4cf1ee40921c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:54.767907 24462 consensus_queue.cc:1035] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:54.768208 24462 consensus_queue.cc:1035] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:54.771696 22634 raft_consensus.cc:1261] T 847f3a2601d44898a14a0b35409981ad P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:54.772230 24592 consensus_queue.cc:1035] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:54.772879 22763 raft_consensus.cc:1261] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:54.775033 22763 raft_consensus.cc:1261] T c2422c666baa48078030c43d1c0938f9 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:54.775032 22633 raft_consensus.cc:1261] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:54.775553 24199 consensus_queue.cc:1035] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:54.775632 24696 consensus_queue.cc:1035] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:54.775861 24199 consensus_queue.cc:1035] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 128.703 ms
time per row: 0.0643515 ms
I20240502 07:58:54.842648 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.133:41661
--local_ip_for_outbound_sockets=127.15.129.133
--tserver_master_addrs=127.15.129.190:35417
--webserver_port=44545
--webserver_interface=127.15.129.133
--builtin_ntp_servers=127.15.129.148:39703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 07:58:54.956672 24739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:54.956908 24739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:54.956954 24739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:54.960201 24739 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:54.960276 24739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:54.960345 24739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.133
I20240502 07:58:54.964049 24739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.133:41661
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.15.129.133
--webserver_port=44545
--tserver_master_addrs=127.15.129.190:35417
--never_fsync=true
--heap_profile_path=/tmp/kudu.24739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.133
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:54.964941 24739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:54.966140 24739 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240502 07:58:54.976262 24739 server_base.cc:1034] running on GCE node
W20240502 07:58:54.977504 24745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:54.978436 24744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:54.978931 24747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:54.979395 24739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:54.980019 24739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:54.981179 24739 hybrid_clock.cc:648] HybridClock initialized: now 1714636734981132 us; error 65 us; skew 500 ppm
I20240502 07:58:54.983366 24739 webserver.cc:460] Webserver started at http://127.15.129.133:44545/ using document root <none> and password file <none>
I20240502 07:58:54.983965 24739 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:54.984033 24739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:54.987310 24739 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20240502 07:58:54.989189 24753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:54.990289 24739 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240502 07:58:54.990391 24739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:54.990805 24739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:55.012012 24739 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:55.012864 24739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:55.013041 24739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:55.013662 24739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:55.014521 24739 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:55.014580 24739 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:55.014626 24739 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:55.014655 24739 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:55.023519 24739 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.133:41661
I20240502 07:58:55.023592 24866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.133:41661 every 8 connection(s)
I20240502 07:58:55.024587 24739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 07:58:55.029050 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 24739
I20240502 07:58:55.029618 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.134:35995
--local_ip_for_outbound_sockets=127.15.129.134
--tserver_master_addrs=127.15.129.190:35417
--webserver_port=46761
--webserver_interface=127.15.129.134
--builtin_ntp_servers=127.15.129.148:39703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:55.034344 24867 heartbeater.cc:347] Connected to a master server at 127.15.129.190:35417
I20240502 07:58:55.034603 24867 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:55.035207 24867 heartbeater.cc:511] Master 127.15.129.190:35417 requested a full tablet report, sending...
I20240502 07:58:55.036446 22225 ts_manager.cc:194] Re-registered known tserver with Master: 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:58:55.037132 22225 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.133:47351
W20240502 07:58:55.148368 24870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:55.148605 24870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:55.148681 24870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:55.151790 24870 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:55.151866 24870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:55.151958 24870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.134
I20240502 07:58:55.155639 24870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.134:35995
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.15.129.134
--webserver_port=46761
--tserver_master_addrs=127.15.129.190:35417
--never_fsync=true
--heap_profile_path=/tmp/kudu.24870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.134
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:55.156561 24870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:55.157724 24870 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:55.166935 24878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:55.168617 24876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:55.169836 24875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:55.170912 24870 server_base.cc:1034] running on GCE node
I20240502 07:58:55.171401 24870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:55.172008 24870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:55.173169 24870 hybrid_clock.cc:648] HybridClock initialized: now 1714636735173145 us; error 36 us; skew 500 ppm
I20240502 07:58:55.175107 24870 webserver.cc:460] Webserver started at http://127.15.129.134:46761/ using document root <none> and password file <none>
I20240502 07:58:55.175678 24870 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:55.175761 24870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:55.179226 24870 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:58:55.181018 24884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:55.182097 24870 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:55.182199 24870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "1aecef81733c47b3b468504934357231"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:55.182680 24870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:55.208801 24870 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:55.209455 24870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:55.209596 24870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:55.210091 24870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:55.211103 24870 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:55.211149 24870 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:55.211189 24870 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:55.211205 24870 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:55.220687 24870 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.134:35995
I20240502 07:58:55.220721 24997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.134:35995 every 8 connection(s)
I20240502 07:58:55.221724 24870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 07:58:55.226218 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 24870
I20240502 07:58:55.226801 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.135:42733
--local_ip_for_outbound_sockets=127.15.129.135
--tserver_master_addrs=127.15.129.190:35417
--webserver_port=33487
--webserver_interface=127.15.129.135
--builtin_ntp_servers=127.15.129.148:39703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:55.231756 24998 heartbeater.cc:347] Connected to a master server at 127.15.129.190:35417
I20240502 07:58:55.232033 24998 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:55.232664 24998 heartbeater.cc:511] Master 127.15.129.190:35417 requested a full tablet report, sending...
I20240502 07:58:55.233707 22225 ts_manager.cc:194] Re-registered known tserver with Master: 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:58:55.234349 22225 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.134:39055
W20240502 07:58:55.346065 25001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:55.346321 25001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:55.346392 25001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:55.349879 25001 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:55.349941 25001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:55.350008 25001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.135
I20240502 07:58:55.353947 25001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.135:42733
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.15.129.135
--webserver_port=33487
--tserver_master_addrs=127.15.129.190:35417
--never_fsync=true
--heap_profile_path=/tmp/kudu.25001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.135
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:55.354825 25001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:55.356215 25001 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:55.364353 25008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:55.365819 25007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:55.367117 25010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:55.368206 25001 server_base.cc:1034] running on GCE node
I20240502 07:58:55.368620 25001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:55.369279 25001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:55.370473 25001 hybrid_clock.cc:648] HybridClock initialized: now 1714636735370445 us; error 48 us; skew 500 ppm
I20240502 07:58:55.372325 25001 webserver.cc:460] Webserver started at http://127.15.129.135:33487/ using document root <none> and password file <none>
I20240502 07:58:55.372865 25001 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:55.372933 25001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:55.376315 25001 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20240502 07:58:55.378190 25016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:55.379418 25001 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:55.379536 25001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "227eec4b5f014d349661f6e0b39b9055"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:55.380033 25001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:55.394680 25001 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:55.395279 25001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:55.395447 25001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:55.395972 25001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:55.396849 25001 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:55.396900 25001 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:55.396943 25001 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:55.396968 25001 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:55.406419 25001 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.135:42733
I20240502 07:58:55.406471 25129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.135:42733 every 8 connection(s)
I20240502 07:58:55.407502 25001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 07:58:55.414508 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 25001
I20240502 07:58:55.414973 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.136:36591
--local_ip_for_outbound_sockets=127.15.129.136
--tserver_master_addrs=127.15.129.190:35417
--webserver_port=46237
--webserver_interface=127.15.129.136
--builtin_ntp_servers=127.15.129.148:39703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:58:55.416443 25130 heartbeater.cc:347] Connected to a master server at 127.15.129.190:35417
I20240502 07:58:55.416709 25130 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:55.417383 25130 heartbeater.cc:511] Master 127.15.129.190:35417 requested a full tablet report, sending...
I20240502 07:58:55.418478 22225 ts_manager.cc:194] Re-registered known tserver with Master: 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:58:55.419329 22225 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.135:48213
W20240502 07:58:55.536597 25133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:55.536805 25133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:58:55.536852 25133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:55.539878 25133 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:58:55.539950 25133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:58:55.540038 25133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.136
I20240502 07:58:55.544001 25133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.136:36591
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.15.129.136
--webserver_port=46237
--tserver_master_addrs=127.15.129.190:35417
--never_fsync=true
--heap_profile_path=/tmp/kudu.25133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.136
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:58:55.545009 25133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:58:55.546382 25133 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:58:55.554728 25141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:55.555830 25139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:58:55.557363 25138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:58:55.558522 25133 server_base.cc:1034] running on GCE node
I20240502 07:58:55.558936 25133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:58:55.559486 25133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:58:55.560688 25133 hybrid_clock.cc:648] HybridClock initialized: now 1714636735560668 us; error 30 us; skew 500 ppm
I20240502 07:58:55.562727 25133 webserver.cc:460] Webserver started at http://127.15.129.136:46237/ using document root <none> and password file <none>
I20240502 07:58:55.563269 25133 fs_manager.cc:359] Metadata directory not provided
I20240502 07:58:55.563333 25133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:58:55.566658 25133 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:58:55.568873 25147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:55.569926 25133 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:58:55.570091 25133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "43930dee0cd5498fa8e357a34dd71432"
format_stamp: "Formatted at 2024-05-02 07:58:39 on dist-test-slave-0hx2"
I20240502 07:58:55.570587 25133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:58:55.587131 25133 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:58:55.587929 25133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:58:55.588079 25133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:58:55.588707 25133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:58:55.589618 25133 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:58:55.589665 25133 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:55.589704 25133 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:58:55.589725 25133 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:58:55.598795 25133 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.136:36591
I20240502 07:58:55.599519 25261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.136:36591 every 8 connection(s)
I20240502 07:58:55.599937 25133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 07:58:55.601590 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 25133
I20240502 07:58:55.605274 22225 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:58:55.616127 25262 heartbeater.cc:347] Connected to a master server at 127.15.129.190:35417
I20240502 07:58:55.616366 25262 heartbeater.cc:464] Registering TS with master...
I20240502 07:58:55.616887 25262 heartbeater.cc:511] Master 127.15.129.190:35417 requested a full tablet report, sending...
I20240502 07:58:55.617926 22225 ts_manager.cc:194] Re-registered known tserver with Master: 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:58:55.618553 22225 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.136:56755
I20240502 07:58:55.620273 25064 tablet_service.cc:1467] Processing CreateTablet for tablet cab832e4a736466aaf674795335f4eae (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b652853fe0964aba8a6fb82cc25028b0]), partition=
I20240502 07:58:55.620982 25064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cab832e4a736466aaf674795335f4eae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:55.623934 24801 tablet_service.cc:1467] Processing CreateTablet for tablet cab832e4a736466aaf674795335f4eae (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b652853fe0964aba8a6fb82cc25028b0]), partition=
I20240502 07:58:55.624672 24801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cab832e4a736466aaf674795335f4eae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:55.628479 25278 tablet_bootstrap.cc:492] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:58:55.630836 25278 tablet_bootstrap.cc:654] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:55.633126 25278 log.cc:824] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:55.634546 25279 tablet_bootstrap.cc:492] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:58:55.636272 25279 tablet_bootstrap.cc:654] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:55.636986 25279 log.cc:824] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:55.638978 25279 tablet_bootstrap.cc:492] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:58:55.639147 25279 ts_tablet_manager.cc:1367] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 07:58:55.639391 25278 tablet_bootstrap.cc:492] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:58:55.639564 25278 ts_tablet_manager.cc:1367] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20240502 07:58:55.642767 25279 raft_consensus.cc:348] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:55.642980 25279 raft_consensus.cc:374] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:55.643055 25279 raft_consensus.cc:729] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:58:55.643469 25279 consensus_queue.cc:260] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:55.643507 25278 raft_consensus.cc:348] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:55.643703 25278 raft_consensus.cc:374] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:55.643757 25278 raft_consensus.cc:729] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:58:55.644210 25279 ts_tablet_manager.cc:1398] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 07:58:55.644177 25278 consensus_queue.cc:260] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:55.644551 24867 heartbeater.cc:503] Master 127.15.129.190:35417 was elected leader, sending a full tablet report...
I20240502 07:58:55.644928 25278 ts_tablet_manager.cc:1398] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 07:58:55.645246 25130 heartbeater.cc:503] Master 127.15.129.190:35417 was elected leader, sending a full tablet report...
I20240502 07:58:55.651343 24932 tablet_service.cc:1467] Processing CreateTablet for tablet cab832e4a736466aaf674795335f4eae (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b652853fe0964aba8a6fb82cc25028b0]), partition=
I20240502 07:58:55.652021 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cab832e4a736466aaf674795335f4eae. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:55.657459 25285 tablet_bootstrap.cc:492] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231: Bootstrap starting.
W20240502 07:58:55.658375 25131 tablet.cc:2369] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:58:55.659951 25285 tablet_bootstrap.cc:654] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:55.660694 25285 log.cc:824] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:55.662462 25285 tablet_bootstrap.cc:492] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:58:55.662647 25285 ts_tablet_manager.cc:1367] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240502 07:58:55.665665 25285 raft_consensus.cc:348] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:55.665890 25285 raft_consensus.cc:374] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:55.665953 25285 raft_consensus.cc:729] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:58:55.666424 25285 consensus_queue.cc:260] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:55.667073 24998 heartbeater.cc:503] Master 127.15.129.190:35417 was elected leader, sending a full tablet report...
I20240502 07:58:55.667237 25285 ts_tablet_manager.cc:1398] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20240502 07:58:55.722805 24999 tablet.cc:2369] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 07:58:55.775864 24868 tablet.cc:2369] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:58:55.783000 25283 raft_consensus.cc:482] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:55.783200 25283 raft_consensus.cc:504] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:55.784512 25283 leader_election.cc:288] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:58:55.789906 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab832e4a736466aaf674795335f4eae" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:58:55.790241 25084 raft_consensus.cc:2454] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:58:55.790341 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab832e4a736466aaf674795335f4eae" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:58:55.790978 24952 raft_consensus.cc:2454] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:58:55.791198 24754 leader_election.cc:302] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:58:55.791466 25283 raft_consensus.cc:2781] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:55.791527 25283 raft_consensus.cc:482] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:55.791558 25283 raft_consensus.cc:3037] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:55.792429 25283 raft_consensus.cc:504] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:55.792799 25283 leader_election.cc:288] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:58:55.793161 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab832e4a736466aaf674795335f4eae" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:58:55.793202 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab832e4a736466aaf674795335f4eae" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:58:55.793299 24952 raft_consensus.cc:3037] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:55.793299 25084 raft_consensus.cc:3037] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:55.794406 24952 raft_consensus.cc:2454] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:58:55.794401 25084 raft_consensus.cc:2454] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:58:55.794754 24757 leader_election.cc:302] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:58:55.795010 25283 raft_consensus.cc:2781] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:55.795241 25283 raft_consensus.cc:686] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:58:55.795589 25283 consensus_queue.cc:237] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:55.798265 22223 catalog_manager.cc:5484] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:55.872859 24801 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:55.872879 25064 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:55.876169 24932 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:55.890410 22612 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:58:55.891012 25195 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:55.904806 22483 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:58:55.905648 22224 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:58:55.922755 22743 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 07:58:55.928754 24952 raft_consensus.cc:1261] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer 4b4e7eeb7f3f42119c0363a79febc54b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:55.929828 25084 raft_consensus.cc:1261] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 4b4e7eeb7f3f42119c0363a79febc54b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:55.930537 25283 consensus_queue.cc:1035] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:55.933992 25283 consensus_queue.cc:1035] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:55.934902 22224 catalog_manager.cc:5576] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733) does not have the latest schema for tablet cab832e4a736466aaf674795335f4eae (table rebalancer_dynamic_table [id=b652853fe0964aba8a6fb82cc25028b0]). Expected version 1 got 0
I20240502 07:58:55.939560 22224 catalog_manager.cc:5576] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995) does not have the latest schema for tablet cab832e4a736466aaf674795335f4eae (table rebalancer_dynamic_table [id=b652853fe0964aba8a6fb82cc25028b0]). Expected version 1 got 0
I20240502 07:58:55.941632 25348 mvcc.cc:205] Tried to move back new op lower bound from 7023152070353555456 to 7023152069821497344. Current Snapshot: MvccSnapshot[applied={T|T < 7023152070353555456}]
I20240502 07:58:55.942451 25351 mvcc.cc:205] Tried to move back new op lower bound from 7023152070353555456 to 7023152069821497344. Current Snapshot: MvccSnapshot[applied={T|T < 7023152070353555456}]
I20240502 07:58:55.942859 22227 catalog_manager.cc:5576] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995) does not have the latest schema for tablet cab832e4a736466aaf674795335f4eae (table rebalancer_dynamic_table [id=b652853fe0964aba8a6fb82cc25028b0]). Expected version 1 got 0
I20240502 07:58:55.943048 25354 tablet.cc:1722] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:58:55.943534 25353 tablet.cc:1722] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:58:55.944058 25354 tablet.cc:1620] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:58:55.945753 25353 tablet.cc:1620] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:58:55.949590 22227 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=b652853fe0964aba8a6fb82cc25028b0] alter complete (version 1)
I20240502 07:58:55.952021 25355 tablet.cc:1722] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:58:55.952996 25355 tablet.cc:1620] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:58:55.953579 25349 mvcc.cc:205] Tried to move back new op lower bound from 7023152070353555456 to 7023152069821497344. Current Snapshot: MvccSnapshot[applied={T|T < 7023152070353555456}]
I20240502 07:58:55.955189 22351 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:58:56.055344 22224 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:58:56.061724 25353 tablet.cc:1722] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:58:56.062068 25353 tablet.cc:1620] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:58:56.062518 25355 tablet.cc:1722] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:58:56.062816 25355 tablet.cc:1620] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:58:56.063202 25354 tablet.cc:1722] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:58:56.063489 25354 tablet.cc:1620] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:58:56.068094 22224 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=b652853fe0964aba8a6fb82cc25028b0] alter complete (version 2)
I20240502 07:58:56.103894 22227 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_0" } modify_external_catalogs: true
I20240502 07:58:56.103927 22221 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:56.104048 22227 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_0" } modify_external_catalogs: true
I20240502 07:58:56.109616 22227 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 8454cd827e5742819d06a9f883d161bb
I20240502 07:58:56.110052 22227 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 41b11b805d4a40c5ab51e5e2ff5b6cf2
I20240502 07:58:56.110282 22227 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 8ffeb8c1919746c3aaeaeb9ddefb8793
I20240502 07:58:56.111035 25195 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:56.111194 22350 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ffeb8c1919746c3aaeaeb9ddefb8793 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:56.111274 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet 8454cd827e5742819d06a9f883d161bb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:56.111397 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet 8454cd827e5742819d06a9f883d161bb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:56.111529 22352 tablet_service.cc:1514] Processing DeleteTablet for tablet 41b11b805d4a40c5ab51e5e2ff5b6cf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:56.111560 22482 tablet_service.cc:1514] Processing DeleteTablet for tablet 41b11b805d4a40c5ab51e5e2ff5b6cf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:56.111896 22743 tablet_service.cc:1514] Processing DeleteTablet for tablet 41b11b805d4a40c5ab51e5e2ff5b6cf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:56.112206 22744 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ffeb8c1919746c3aaeaeb9ddefb8793 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:56.113394 22614 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ffeb8c1919746c3aaeaeb9ddefb8793 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:58:56.113399 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet 8454cd827e5742819d06a9f883d161bb with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:58:56.114593 25417 tablet_replica.cc:330] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:56.114691 25064 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:56.114925 25417 raft_consensus.cc:2229] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:56.118373 24801 tablet_service.cc:1467] Processing CreateTablet for tablet 4017b918d5f1485aaaf05c764f08ad67 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=ad840b2a5e7149b78acd24dc98f05685]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.118749 24801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4017b918d5f1485aaaf05c764f08ad67. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.120925 25279 tablet_bootstrap.cc:492] T 4017b918d5f1485aaaf05c764f08ad67 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:58:56.121156 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 9f916726456f4aa8936a9366468bc471 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=ad840b2a5e7149b78acd24dc98f05685]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.121383 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f916726456f4aa8936a9366468bc471. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.121793 25279 tablet_bootstrap.cc:654] T 4017b918d5f1485aaaf05c764f08ad67 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.123582 22482 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:58:56.125212 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 4017b918d5f1485aaaf05c764f08ad67 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=ad840b2a5e7149b78acd24dc98f05685]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.125144 25064 tablet_service.cc:1467] Processing CreateTablet for tablet 9f916726456f4aa8936a9366468bc471 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=ad840b2a5e7149b78acd24dc98f05685]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.125491 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4017b918d5f1485aaaf05c764f08ad67. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.125604 25064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f916726456f4aa8936a9366468bc471. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.129076 22744 tablet_service.cc:1467] Processing CreateTablet for tablet 9f916726456f4aa8936a9366468bc471 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=ad840b2a5e7149b78acd24dc98f05685]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.129376 22744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f916726456f4aa8936a9366468bc471. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.129364 24932 tablet_service.cc:1467] Processing CreateTablet for tablet a3ffa0ad72d7493082f697a1215bdbf4 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=ad840b2a5e7149b78acd24dc98f05685]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.129694 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3ffa0ad72d7493082f697a1215bdbf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.131880 25285 tablet_bootstrap.cc:492] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:58:56.132566 24800 tablet_service.cc:1467] Processing CreateTablet for tablet a3ffa0ad72d7493082f697a1215bdbf4 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=ad840b2a5e7149b78acd24dc98f05685]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.132781 25285 tablet_bootstrap.cc:654] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.132833 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3ffa0ad72d7493082f697a1215bdbf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.134902 25278 tablet_bootstrap.cc:492] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:58:56.135686 25418 tablet_replica.cc:330] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:56.135787 25278 tablet_bootstrap.cc:654] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.136021 25418 raft_consensus.cc:2229] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:56.136209 25418 raft_consensus.cc:2258] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.137213 25418 ts_tablet_manager.cc:1875] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.138475 25419 tablet_replica.cc:330] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:58:56.139582 25419 raft_consensus.cc:2229] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:56.139940 25419 raft_consensus.cc:2258] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.141074 25419 ts_tablet_manager.cc:1875] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.142277 24800 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:56.144438 25194 tablet_service.cc:1467] Processing CreateTablet for tablet a3ffa0ad72d7493082f697a1215bdbf4 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=ad840b2a5e7149b78acd24dc98f05685]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.145104 25195 tablet_service.cc:1467] Processing CreateTablet for tablet 4017b918d5f1485aaaf05c764f08ad67 (DEFAULT_TABLE table=rebalancer_extra_table_0 [id=ad840b2a5e7149b78acd24dc98f05685]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:56.145313 25419 ts_tablet_manager.cc:1888] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:56.145416 25419 log.cc:1196] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/8ffeb8c1919746c3aaeaeb9ddefb8793
I20240502 07:58:56.145479 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3ffa0ad72d7493082f697a1215bdbf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.146082 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4017b918d5f1485aaaf05c764f08ad67. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.147193 25418 ts_tablet_manager.cc:1888] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:56.147336 25418 log.cc:1196] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/41b11b805d4a40c5ab51e5e2ff5b6cf2
I20240502 07:58:56.147715 25418 ts_tablet_manager.cc:1909] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:56.148031 25419 ts_tablet_manager.cc:1909] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:58:56.149245 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 41b11b805d4a40c5ab51e5e2ff5b6cf2 (table rebalance_test_table_0 [id=bc21302f92b248cf8c6d0a8d666ab5a4]) successfully deleted
I20240502 07:58:56.150219 24932 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:56.150887 25419 tablet_replica.cc:330] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:58:56.150949 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 8ffeb8c1919746c3aaeaeb9ddefb8793 (table rebalance_test_table_0 [id=bc21302f92b248cf8c6d0a8d666ab5a4]) successfully deleted
I20240502 07:58:56.151141 25418 tablet_replica.cc:330] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:56.151340 25418 raft_consensus.cc:2229] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:56.151536 25419 raft_consensus.cc:2229] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:56.151742 25418 raft_consensus.cc:2258] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.151930 25419 raft_consensus.cc:2258] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.152573 25418 ts_tablet_manager.cc:1875] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.153038 25421 tablet_bootstrap.cc:492] T 4017b918d5f1485aaaf05c764f08ad67 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:56.153184 25419 ts_tablet_manager.cc:1875] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.152690 22744 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 07:58:56.154050 25422 tablet_bootstrap.cc:492] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:56.154924 25421 tablet_bootstrap.cc:654] T 4017b918d5f1485aaaf05c764f08ad67 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.155115 25422 tablet_bootstrap.cc:654] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.156075 25421 tablet_bootstrap.cc:492] T 4017b918d5f1485aaaf05c764f08ad67 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:56.156167 25421 ts_tablet_manager.cc:1367] T 4017b918d5f1485aaaf05c764f08ad67 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:56.156571 25421 raft_consensus.cc:348] T 4017b918d5f1485aaaf05c764f08ad67 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:56.156682 25421 raft_consensus.cc:374] T 4017b918d5f1485aaaf05c764f08ad67 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.156719 25421 raft_consensus.cc:729] T 4017b918d5f1485aaaf05c764f08ad67 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.156848 25421 consensus_queue.cc:260] T 4017b918d5f1485aaaf05c764f08ad67 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:56.157145 25421 ts_tablet_manager.cc:1398] T 4017b918d5f1485aaaf05c764f08ad67 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:56.159600 22612 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:58:56.164129 25426 tablet_bootstrap.cc:492] T a3ffa0ad72d7493082f697a1215bdbf4 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:58:56.166551 25426 tablet_bootstrap.cc:654] T a3ffa0ad72d7493082f697a1215bdbf4 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.167367 25426 log.cc:824] T a3ffa0ad72d7493082f697a1215bdbf4 P 43930dee0cd5498fa8e357a34dd71432: Log is configured to *not* fsync() on all Append() calls
I20240502 07:58:56.169186 25426 tablet_bootstrap.cc:492] T a3ffa0ad72d7493082f697a1215bdbf4 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:58:56.169400 25426 ts_tablet_manager.cc:1367] T a3ffa0ad72d7493082f697a1215bdbf4 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20240502 07:58:56.171516 22221 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:58:56.173386 22352 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:58:56.179700 25353 tablet.cc:1722] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:58:56.181023 25416 tablet_replica.cc:330] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:56.181397 25417 raft_consensus.cc:2258] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.181808 25355 tablet.cc:1722] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:58:56.182039 25416 raft_consensus.cc:2229] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:56.182351 25416 raft_consensus.cc:2258] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.183245 25417 ts_tablet_manager.cc:1875] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.186789 25416 ts_tablet_manager.cc:1875] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.187858 25354 tablet.cc:1722] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:58:56.188297 25285 tablet_bootstrap.cc:492] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:58:56.188372 25285 ts_tablet_manager.cc:1367] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.057s user 0.001s sys 0.000s
I20240502 07:58:56.188941 25285 raft_consensus.cc:348] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.189065 25285 raft_consensus.cc:374] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.189107 25285 raft_consensus.cc:729] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.189211 25285 consensus_queue.cc:260] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.189502 25285 ts_tablet_manager.cc:1398] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:56.190117 25278 tablet_bootstrap.cc:492] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:58:56.190208 25278 ts_tablet_manager.cc:1367] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.055s user 0.002s sys 0.000s
I20240502 07:58:56.190122 25426 raft_consensus.cc:348] T a3ffa0ad72d7493082f697a1215bdbf4 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.190371 25426 raft_consensus.cc:374] T a3ffa0ad72d7493082f697a1215bdbf4 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.190434 25426 raft_consensus.cc:729] T a3ffa0ad72d7493082f697a1215bdbf4 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.190802 25278 raft_consensus.cc:348] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:56.190910 25278 raft_consensus.cc:374] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.190946 25278 raft_consensus.cc:729] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.191064 25278 consensus_queue.cc:260] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:56.191318 25278 ts_tablet_manager.cc:1398] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:56.192011 25426 consensus_queue.cc:260] T a3ffa0ad72d7493082f697a1215bdbf4 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.192442 22227 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=b652853fe0964aba8a6fb82cc25028b0] alter complete (version 3)
I20240502 07:58:56.192924 25426 ts_tablet_manager.cc:1398] T a3ffa0ad72d7493082f697a1215bdbf4 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.023s user 0.005s sys 0.000s
I20240502 07:58:56.193457 25279 tablet_bootstrap.cc:492] T 4017b918d5f1485aaaf05c764f08ad67 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:58:56.193557 25279 ts_tablet_manager.cc:1367] T 4017b918d5f1485aaaf05c764f08ad67 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.073s user 0.002s sys 0.000s
I20240502 07:58:56.194186 25279 raft_consensus.cc:348] T 4017b918d5f1485aaaf05c764f08ad67 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:56.194294 25279 raft_consensus.cc:374] T 4017b918d5f1485aaaf05c764f08ad67 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.194327 25279 raft_consensus.cc:729] T 4017b918d5f1485aaaf05c764f08ad67 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.194432 25279 consensus_queue.cc:260] T 4017b918d5f1485aaaf05c764f08ad67 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:56.194705 25279 ts_tablet_manager.cc:1398] T 4017b918d5f1485aaaf05c764f08ad67 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:56.194895 25279 tablet_bootstrap.cc:492] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:58:56.195487 25262 heartbeater.cc:503] Master 127.15.129.190:35417 was elected leader, sending a full tablet report...
I20240502 07:58:56.195900 25279 tablet_bootstrap.cc:654] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.199265 25426 tablet_bootstrap.cc:492] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:58:56.199275 25417 ts_tablet_manager.cc:1888] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:56.199371 25417 log.cc:1196] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/8454cd827e5742819d06a9f883d161bb
I20240502 07:58:56.199568 25419 ts_tablet_manager.cc:1888] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:56.201121 25417 ts_tablet_manager.cc:1909] T 8454cd827e5742819d06a9f883d161bb P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:58:56.201313 25426 tablet_bootstrap.cc:654] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.202261 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 8454cd827e5742819d06a9f883d161bb (table rebalance_test_table_0 [id=bc21302f92b248cf8c6d0a8d666ab5a4]) successfully deleted
I20240502 07:58:56.202522 25416 ts_tablet_manager.cc:1888] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:56.202543 25418 ts_tablet_manager.cc:1888] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:56.202616 25418 log.cc:1196] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/8ffeb8c1919746c3aaeaeb9ddefb8793
I20240502 07:58:56.202694 25419 log.cc:1196] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/8454cd827e5742819d06a9f883d161bb
I20240502 07:58:56.202711 25416 log.cc:1196] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/8ffeb8c1919746c3aaeaeb9ddefb8793
I20240502 07:58:56.203051 25419 ts_tablet_manager.cc:1909] T 8454cd827e5742819d06a9f883d161bb P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:58:56.203073 25416 ts_tablet_manager.cc:1909] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:56.203377 25418 ts_tablet_manager.cc:1909] T 8ffeb8c1919746c3aaeaeb9ddefb8793 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:56.204553 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 8ffeb8c1919746c3aaeaeb9ddefb8793 (table rebalance_test_table_0 [id=bc21302f92b248cf8c6d0a8d666ab5a4]) successfully deleted
I20240502 07:58:56.204866 25416 tablet_replica.cc:330] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:56.205032 25417 tablet_replica.cc:330] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:56.205197 25417 raft_consensus.cc:2229] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:56.205386 25417 raft_consensus.cc:2258] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.205945 25417 ts_tablet_manager.cc:1875] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.206019 25279 tablet_bootstrap.cc:492] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:58:56.206126 25279 ts_tablet_manager.cc:1367] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 07:58:56.206204 25416 raft_consensus.cc:2229] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:56.206357 25416 raft_consensus.cc:2258] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.206415 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 8ffeb8c1919746c3aaeaeb9ddefb8793 (table rebalance_test_table_0 [id=bc21302f92b248cf8c6d0a8d666ab5a4]) successfully deleted
I20240502 07:58:56.206597 25279 raft_consensus.cc:348] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:56.206732 25279 raft_consensus.cc:374] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.206771 25279 raft_consensus.cc:729] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.206905 25279 consensus_queue.cc:260] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:56.207187 25279 ts_tablet_manager.cc:1398] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:56.207334 25416 ts_tablet_manager.cc:1875] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.207373 25279 tablet_bootstrap.cc:492] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:58:56.207918 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 8454cd827e5742819d06a9f883d161bb (table rebalance_test_table_0 [id=bc21302f92b248cf8c6d0a8d666ab5a4]) successfully deleted
I20240502 07:58:56.208221 25279 tablet_bootstrap.cc:654] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.211704 25416 ts_tablet_manager.cc:1888] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:56.211776 25422 tablet_bootstrap.cc:492] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:56.211884 25417 ts_tablet_manager.cc:1888] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:56.211930 25422 ts_tablet_manager.cc:1367] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.058s user 0.002s sys 0.000s
I20240502 07:58:56.212007 25417 log.cc:1196] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/41b11b805d4a40c5ab51e5e2ff5b6cf2
I20240502 07:58:56.212446 25416 log.cc:1196] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/8454cd827e5742819d06a9f883d161bb
I20240502 07:58:56.212332 25417 ts_tablet_manager.cc:1909] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:58:56.212507 25422 raft_consensus.cc:348] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:56.212677 25422 raft_consensus.cc:374] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.212755 25422 raft_consensus.cc:729] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.212838 25416 ts_tablet_manager.cc:1909] T 8454cd827e5742819d06a9f883d161bb P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:56.212997 25422 consensus_queue.cc:260] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:56.213716 25422 ts_tablet_manager.cc:1398] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:56.214633 25279 tablet_bootstrap.cc:492] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:58:56.214764 25279 ts_tablet_manager.cc:1367] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:58:56.214864 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 41b11b805d4a40c5ab51e5e2ff5b6cf2 (table rebalance_test_table_0 [id=bc21302f92b248cf8c6d0a8d666ab5a4]) successfully deleted
I20240502 07:58:56.214967 25416 tablet_replica.cc:330] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:56.215087 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 8454cd827e5742819d06a9f883d161bb (table rebalance_test_table_0 [id=bc21302f92b248cf8c6d0a8d666ab5a4]) successfully deleted
I20240502 07:58:56.215116 25416 raft_consensus.cc:2229] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:56.215399 25279 raft_consensus.cc:348] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.215564 25279 raft_consensus.cc:374] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.215638 25279 raft_consensus.cc:729] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.215795 25279 consensus_queue.cc:260] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.216133 25279 ts_tablet_manager.cc:1398] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:56.216228 25416 raft_consensus.cc:2258] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.216684 25416 ts_tablet_manager.cc:1875] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.219234 25426 tablet_bootstrap.cc:492] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:58:56.219331 25426 ts_tablet_manager.cc:1367] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240502 07:58:56.219815 25426 raft_consensus.cc:348] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:56.219918 25426 raft_consensus.cc:374] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.219952 25426 raft_consensus.cc:729] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.220088 25426 consensus_queue.cc:260] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:56.225417 25426 ts_tablet_manager.cc:1398] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 07:58:56.225687 25416 ts_tablet_manager.cc:1888] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:56.225772 25416 log.cc:1196] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/41b11b805d4a40c5ab51e5e2ff5b6cf2
I20240502 07:58:56.226114 25416 ts_tablet_manager.cc:1909] T 41b11b805d4a40c5ab51e5e2ff5b6cf2 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:56.226904 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 41b11b805d4a40c5ab51e5e2ff5b6cf2 (table rebalance_test_table_0 [id=bc21302f92b248cf8c6d0a8d666ab5a4]) successfully deleted
I20240502 07:58:56.247438 25403 raft_consensus.cc:482] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:56.247565 25403 raft_consensus.cc:504] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:56.248154 25403 leader_election.cc:288] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:56.248684 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f916726456f4aa8936a9366468bc471" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:58:56.248837 25084 raft_consensus.cc:2454] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:58:56.249290 24754 leader_election.cc:302] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:58:56.249481 25403 raft_consensus.cc:2781] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:56.249527 25403 raft_consensus.cc:482] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:56.249564 25403 raft_consensus.cc:3037] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.250701 25403 raft_consensus.cc:504] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:56.251068 25403 leader_election.cc:288] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:56.251367 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f916726456f4aa8936a9366468bc471" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:58:56.251490 25084 raft_consensus.cc:3037] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.252377 25084 raft_consensus.cc:2454] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:58:56.252449 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f916726456f4aa8936a9366468bc471" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:56.252578 22764 raft_consensus.cc:3037] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.252404 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f916726456f4aa8936a9366468bc471" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:56.253592 24754 leader_election.cc:302] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:58:56.253742 22764 raft_consensus.cc:2454] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:58:56.253784 25403 raft_consensus.cc:2781] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:56.253897 25403 raft_consensus.cc:686] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:58:56.254045 25403 consensus_queue.cc:237] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:56.255683 22221 catalog_manager.cc:5484] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:58:56.299088 22221 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:58:56.299214 22221 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:58:56.301554 22221 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet cab832e4a736466aaf674795335f4eae
I20240502 07:58:56.302637 24800 tablet_service.cc:1514] Processing DeleteTablet for tablet cab832e4a736466aaf674795335f4eae with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:58:56.302637 25064 tablet_service.cc:1514] Processing DeleteTablet for tablet cab832e4a736466aaf674795335f4eae with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:58:56.303107 25456 tablet_replica.cc:330] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:58:56.303112 24932 tablet_service.cc:1514] Processing DeleteTablet for tablet cab832e4a736466aaf674795335f4eae with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:37828
I20240502 07:58:56.303431 25456 raft_consensus.cc:2229] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:56.303519 25458 tablet_replica.cc:330] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:58:56.303747 25458 raft_consensus.cc:2229] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:56.303807 25456 raft_consensus.cc:2258] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.303957 25458 raft_consensus.cc:2258] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.304109 25457 tablet_replica.cc:330] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:58:56.304334 25457 raft_consensus.cc:2229] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:56.304522 25457 raft_consensus.cc:2258] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.305609 25457 ts_tablet_manager.cc:1875] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.305630 25458 ts_tablet_manager.cc:1875] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.305855 25456 ts_tablet_manager.cc:1875] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.307986 25458 ts_tablet_manager.cc:1888] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:56.308079 25458 log.cc:1196] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/cab832e4a736466aaf674795335f4eae
I20240502 07:58:56.308298 25457 ts_tablet_manager.cc:1888] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:56.308405 25458 ts_tablet_manager.cc:1909] T cab832e4a736466aaf674795335f4eae P 1aecef81733c47b3b468504934357231: Deleting consensus metadata
I20240502 07:58:56.309613 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet cab832e4a736466aaf674795335f4eae (table rebalancer_dynamic_table_new_name [id=b652853fe0964aba8a6fb82cc25028b0]) successfully deleted
I20240502 07:58:56.310722 25457 log.cc:1196] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/cab832e4a736466aaf674795335f4eae
I20240502 07:58:56.311123 25457 ts_tablet_manager.cc:1909] T cab832e4a736466aaf674795335f4eae P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:58:56.311905 25456 ts_tablet_manager.cc:1888] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:56.311998 25456 log.cc:1196] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/cab832e4a736466aaf674795335f4eae
I20240502 07:58:56.312276 25456 ts_tablet_manager.cc:1909] T cab832e4a736466aaf674795335f4eae P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:58:56.315116 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet cab832e4a736466aaf674795335f4eae (table rebalancer_dynamic_table_new_name [id=b652853fe0964aba8a6fb82cc25028b0]) successfully deleted
I20240502 07:58:56.315373 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet cab832e4a736466aaf674795335f4eae (table rebalancer_dynamic_table_new_name [id=b652853fe0964aba8a6fb82cc25028b0]) successfully deleted
W20240502 07:58:56.351181 25263 tablet.cc:2369] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:58:56.403923 22226 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:58:56.409458 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 1478893ea6d54d5b80e19396196a3dd3 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4fe10cc726b74369a67b32764fc4261b]), partition=
I20240502 07:58:56.409709 22482 tablet_service.cc:1467] Processing CreateTablet for tablet 1478893ea6d54d5b80e19396196a3dd3 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4fe10cc726b74369a67b32764fc4261b]), partition=
I20240502 07:58:56.409763 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1478893ea6d54d5b80e19396196a3dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.409988 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1478893ea6d54d5b80e19396196a3dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.411747 25279 tablet_bootstrap.cc:492] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:58:56.412286 22612 tablet_service.cc:1467] Processing CreateTablet for tablet 1478893ea6d54d5b80e19396196a3dd3 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=4fe10cc726b74369a67b32764fc4261b]), partition=
I20240502 07:58:56.412537 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1478893ea6d54d5b80e19396196a3dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:56.412691 25279 tablet_bootstrap.cc:654] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.412705 25479 tablet_bootstrap.cc:492] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:56.413578 25479 tablet_bootstrap.cc:654] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.414608 25479 tablet_bootstrap.cc:492] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:56.414732 25479 ts_tablet_manager.cc:1367] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:56.415004 25482 tablet_bootstrap.cc:492] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:56.415200 25479 raft_consensus.cc:348] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.415314 25479 raft_consensus.cc:374] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.415351 25479 raft_consensus.cc:729] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.415468 25479 consensus_queue.cc:260] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.415782 25479 ts_tablet_manager.cc:1398] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:56.415812 25482 tablet_bootstrap.cc:654] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:56.417387 25482 tablet_bootstrap.cc:492] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:56.417495 25482 ts_tablet_manager.cc:1367] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:56.417933 25482 raft_consensus.cc:348] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.418071 25482 raft_consensus.cc:374] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.418126 25482 raft_consensus.cc:729] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.418282 25482 consensus_queue.cc:260] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.418601 25482 ts_tablet_manager.cc:1398] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:56.422508 25279 tablet_bootstrap.cc:492] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:58:56.422628 25279 ts_tablet_manager.cc:1367] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 07:58:56.423082 25279 raft_consensus.cc:348] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.423197 25279 raft_consensus.cc:374] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:56.423228 25279 raft_consensus.cc:729] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:58:56.423341 25279 consensus_queue.cc:260] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.423605 25279 ts_tablet_manager.cc:1398] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:56.426103 25287 raft_consensus.cc:482] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:56.426227 25287 raft_consensus.cc:504] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.427395 25287 leader_election.cc:288] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:58:56.431779 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3ffa0ad72d7493082f697a1215bdbf4" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:58:56.432092 25215 raft_consensus.cc:2454] T a3ffa0ad72d7493082f697a1215bdbf4 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:58:56.432518 24888 leader_election.cc:302] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:58:56.432787 25287 raft_consensus.cc:2781] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:56.432857 25287 raft_consensus.cc:482] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:56.432896 25287 raft_consensus.cc:3037] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.433857 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3ffa0ad72d7493082f697a1215bdbf4" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:58:56.434010 25287 raft_consensus.cc:504] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.434079 24821 raft_consensus.cc:2454] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:58:56.434374 25287 leader_election.cc:288] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:58:56.434695 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3ffa0ad72d7493082f697a1215bdbf4" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:58:56.434813 24821 raft_consensus.cc:3037] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.434906 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3ffa0ad72d7493082f697a1215bdbf4" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:58:56.435029 25215 raft_consensus.cc:3037] T a3ffa0ad72d7493082f697a1215bdbf4 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.435930 24821 raft_consensus.cc:2454] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:58:56.436111 25215 raft_consensus.cc:2454] T a3ffa0ad72d7493082f697a1215bdbf4 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:58:56.436280 24887 leader_election.cc:302] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:58:56.436580 25287 raft_consensus.cc:2781] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:56.436815 25287 raft_consensus.cc:686] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:58:56.437713 25287 consensus_queue.cc:237] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.440187 22227 catalog_manager.cc:5484] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:56.462939 25429 raft_consensus.cc:482] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:56.463125 25429 raft_consensus.cc:504] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:56.464354 25429 leader_election.cc:288] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:56.468402 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4017b918d5f1485aaaf05c764f08ad67" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:58:56.468564 24821 raft_consensus.cc:2454] T 4017b918d5f1485aaaf05c764f08ad67 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:58:56.468947 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4017b918d5f1485aaaf05c764f08ad67" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:56.468993 25150 leader_election.cc:302] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:58:56.469087 22372 raft_consensus.cc:2454] T 4017b918d5f1485aaaf05c764f08ad67 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:58:56.469300 25429 raft_consensus.cc:2781] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:56.469405 25429 raft_consensus.cc:482] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:56.469461 25429 raft_consensus.cc:3037] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.470396 25429 raft_consensus.cc:504] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:56.470768 25429 leader_election.cc:288] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:56.471063 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4017b918d5f1485aaaf05c764f08ad67" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:58:56.471096 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4017b918d5f1485aaaf05c764f08ad67" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:56.471202 24821 raft_consensus.cc:3037] T 4017b918d5f1485aaaf05c764f08ad67 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.471217 22372 raft_consensus.cc:3037] T 4017b918d5f1485aaaf05c764f08ad67 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.472151 22372 raft_consensus.cc:2454] T 4017b918d5f1485aaaf05c764f08ad67 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:58:56.472151 24821 raft_consensus.cc:2454] T 4017b918d5f1485aaaf05c764f08ad67 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:58:56.472460 25150 leader_election.cc:302] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:58:56.472659 25429 raft_consensus.cc:2781] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:56.472770 25429 raft_consensus.cc:686] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:58:56.473074 25429 consensus_queue.cc:237] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:56.475762 25256 raft_consensus.cc:482] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:56.475868 25256 raft_consensus.cc:504] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.476334 25256 leader_election.cc:288] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:58:56.476756 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1478893ea6d54d5b80e19396196a3dd3" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:58:56.476886 22502 raft_consensus.cc:2454] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:56.477020 22227 catalog_manager.cc:5484] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:56.477792 22567 leader_election.cc:302] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:56.478037 25256 raft_consensus.cc:2781] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:56.478088 25256 raft_consensus.cc:482] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:56.478142 25256 raft_consensus.cc:3037] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.479116 25256 raft_consensus.cc:504] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.479491 25256 leader_election.cc:288] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:58:56.479992 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1478893ea6d54d5b80e19396196a3dd3" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:58:56.480134 22502 raft_consensus.cc:3037] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.481247 22502 raft_consensus.cc:2454] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:56.481622 22567 leader_election.cc:302] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:56.483716 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1478893ea6d54d5b80e19396196a3dd3" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:58:56.483777 25256 raft_consensus.cc:2781] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:56.483873 24821 raft_consensus.cc:2454] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:56.483954 25256 raft_consensus.cc:686] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:56.483954 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1478893ea6d54d5b80e19396196a3dd3" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:58:56.484056 24820 raft_consensus.cc:3037] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:56.484117 25256 consensus_queue.cc:237] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.485144 24820 raft_consensus.cc:2454] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:56.486768 22221 catalog_manager.cc:5484] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:56.499845 24932 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:56.499928 25064 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:56.502748 24800 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:56.518563 22612 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 07:58:56.522567 25195 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:56.537889 22352 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:58:56.543246 22482 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:58:56.551944 22744 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 07:58:56.595852 22221 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:58:56.601284 22503 raft_consensus.cc:1261] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:56.601374 24820 raft_consensus.cc:1261] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:56.602130 24462 consensus_queue.cc:1035] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.602797 24462 consensus_queue.cc:1035] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.605546 22226 catalog_manager.cc:5576] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661) does not have the latest schema for tablet 1478893ea6d54d5b80e19396196a3dd3 (table rebalancer_dynamic_table [id=4fe10cc726b74369a67b32764fc4261b]). Expected version 1 got 0
I20240502 07:58:56.606107 22227 catalog_manager.cc:5576] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625) does not have the latest schema for tablet 1478893ea6d54d5b80e19396196a3dd3 (table rebalancer_dynamic_table [id=4fe10cc726b74369a67b32764fc4261b]). Expected version 1 got 0
I20240502 07:58:56.607568 25513 tablet.cc:1722] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:58:56.608364 25513 tablet.cc:1620] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
I20240502 07:58:56.608669 25353 tablet.cc:1722] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:58:56.608960 25353 tablet.cc:1620] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:58:56.613062 22221 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_2" } modify_external_catalogs: true
I20240502 07:58:56.613219 22221 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_2" } modify_external_catalogs: true
I20240502 07:58:56.615345 22226 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=4fe10cc726b74369a67b32764fc4261b] alter complete (version 1)
I20240502 07:58:56.619908 22222 catalog_manager.cc:5576] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625) does not have the latest schema for tablet 1478893ea6d54d5b80e19396196a3dd3 (table rebalancer_dynamic_table [id=4fe10cc726b74369a67b32764fc4261b]). Expected version 1 got 0
I20240502 07:58:56.621063 22221 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet b005f3f379ba43209b105632ba773165
I20240502 07:58:56.621385 22221 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet dc9be64f2ead4130a9f1f0694131640f
I20240502 07:58:56.621577 22221 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 0c6512754f394ad2a6bca0bd276a76b8
I20240502 07:58:56.621713 22482 tablet_service.cc:1514] Processing DeleteTablet for tablet b005f3f379ba43209b105632ba773165 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:56.621897 25417 tablet_replica.cc:330] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:56.622041 25417 raft_consensus.cc:2229] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:56.622184 25417 raft_consensus.cc:2258] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.622503 22743 tablet_service.cc:1514] Processing DeleteTablet for tablet dc9be64f2ead4130a9f1f0694131640f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:56.622546 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c6512754f394ad2a6bca0bd276a76b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:56.622669 25418 tablet_replica.cc:330] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:56.622715 25417 ts_tablet_manager.cc:1875] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.622743 22350 tablet_service.cc:1514] Processing DeleteTablet for tablet dc9be64f2ead4130a9f1f0694131640f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:56.622786 25418 raft_consensus.cc:2229] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:56.622807 22352 tablet_service.cc:1514] Processing DeleteTablet for tablet b005f3f379ba43209b105632ba773165 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:56.622923 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c6512754f394ad2a6bca0bd276a76b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:56.623028 25418 raft_consensus.cc:2258] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.623188 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet dc9be64f2ead4130a9f1f0694131640f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:58:56.623349 25419 tablet_replica.cc:330] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:58:56.623345 25520 tablet.cc:1722] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:58:56.623459 25419 raft_consensus.cc:2229] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:56.623518 25418 ts_tablet_manager.cc:1875] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.623574 25419 raft_consensus.cc:2258] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.624030 25419 ts_tablet_manager.cc:1875] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.624085 25520 tablet.cc:1620] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c: MemRowSet was empty: no flush needed.
I20240502 07:58:56.624882 25417 ts_tablet_manager.cc:1888] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:56.624944 25417 log.cc:1196] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/b005f3f379ba43209b105632ba773165
I20240502 07:58:56.625205 25417 ts_tablet_manager.cc:1909] T b005f3f379ba43209b105632ba773165 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:58:56.625499 22744 tablet_service.cc:1514] Processing DeleteTablet for tablet b005f3f379ba43209b105632ba773165 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:56.625630 25418 ts_tablet_manager.cc:1888] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:56.625692 25418 log.cc:1196] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/dc9be64f2ead4130a9f1f0694131640f
I20240502 07:58:56.625948 25418 ts_tablet_manager.cc:1909] T dc9be64f2ead4130a9f1f0694131640f P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:56.626052 22482 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c6512754f394ad2a6bca0bd276a76b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:56.626161 25416 tablet_replica.cc:330] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:56.626315 25416 raft_consensus.cc:2229] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:56.626343 25419 ts_tablet_manager.cc:1888] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:56.626401 25419 log.cc:1196] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/dc9be64f2ead4130a9f1f0694131640f
I20240502 07:58:56.626474 25416 raft_consensus.cc:2258] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.626657 25419 ts_tablet_manager.cc:1909] T dc9be64f2ead4130a9f1f0694131640f P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:58:56.626713 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet dc9be64f2ead4130a9f1f0694131640f (table rebalance_test_table_2 [id=0937fa4568b0489ebd23bdd0b698a759]) successfully deleted
I20240502 07:58:56.626912 25418 tablet_replica.cc:330] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:56.626998 25416 ts_tablet_manager.cc:1875] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.627089 25418 raft_consensus.cc:2229] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:56.627331 25418 raft_consensus.cc:2258] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.628252 25417 tablet_replica.cc:330] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:56.628441 25417 raft_consensus.cc:2229] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:56.628671 25417 raft_consensus.cc:2258] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.628881 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet dc9be64f2ead4130a9f1f0694131640f (table rebalance_test_table_2 [id=0937fa4568b0489ebd23bdd0b698a759]) successfully deleted
I20240502 07:58:56.628919 25416 ts_tablet_manager.cc:1888] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:56.628981 25416 log.cc:1196] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/dc9be64f2ead4130a9f1f0694131640f
I20240502 07:58:56.629240 25416 ts_tablet_manager.cc:1909] T dc9be64f2ead4130a9f1f0694131640f P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:56.629289 25418 ts_tablet_manager.cc:1875] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.629398 25417 ts_tablet_manager.cc:1875] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.629587 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet b005f3f379ba43209b105632ba773165 (table rebalance_test_table_2 [id=0937fa4568b0489ebd23bdd0b698a759]) successfully deleted
I20240502 07:58:56.629926 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet dc9be64f2ead4130a9f1f0694131640f (table rebalance_test_table_2 [id=0937fa4568b0489ebd23bdd0b698a759]) successfully deleted
I20240502 07:58:56.630118 25416 tablet_replica.cc:330] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:56.630241 25416 raft_consensus.cc:2229] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:56.630374 25416 raft_consensus.cc:2258] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.630908 25416 ts_tablet_manager.cc:1875] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.631440 25417 ts_tablet_manager.cc:1888] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:56.631517 25417 log.cc:1196] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/0c6512754f394ad2a6bca0bd276a76b8
I20240502 07:58:56.631763 25417 ts_tablet_manager.cc:1909] T 0c6512754f394ad2a6bca0bd276a76b8 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:58:56.632167 25418 ts_tablet_manager.cc:1888] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:56.632243 25418 log.cc:1196] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/b005f3f379ba43209b105632ba773165
I20240502 07:58:56.632567 25418 ts_tablet_manager.cc:1909] T b005f3f379ba43209b105632ba773165 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:56.632571 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 0c6512754f394ad2a6bca0bd276a76b8 (table rebalance_test_table_2 [id=0937fa4568b0489ebd23bdd0b698a759]) successfully deleted
I20240502 07:58:56.632591 25416 ts_tablet_manager.cc:1888] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:56.632653 25416 log.cc:1196] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/b005f3f379ba43209b105632ba773165
I20240502 07:58:56.632905 25416 ts_tablet_manager.cc:1909] T b005f3f379ba43209b105632ba773165 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:56.633093 25418 tablet_replica.cc:330] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:56.633153 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet b005f3f379ba43209b105632ba773165 (table rebalance_test_table_2 [id=0937fa4568b0489ebd23bdd0b698a759]) successfully deleted
I20240502 07:58:56.633265 25418 raft_consensus.cc:2229] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:56.633414 25418 raft_consensus.cc:2258] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.633437 25416 tablet_replica.cc:330] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:56.633512 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet b005f3f379ba43209b105632ba773165 (table rebalance_test_table_2 [id=0937fa4568b0489ebd23bdd0b698a759]) successfully deleted
I20240502 07:58:56.633562 25416 raft_consensus.cc:2229] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:56.633695 25416 raft_consensus.cc:2258] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.634002 25418 ts_tablet_manager.cc:1875] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.634276 25416 ts_tablet_manager.cc:1875] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.635779 25416 ts_tablet_manager.cc:1888] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:56.635834 25416 log.cc:1196] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/0c6512754f394ad2a6bca0bd276a76b8
I20240502 07:58:56.635859 25418 ts_tablet_manager.cc:1888] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:56.635915 25418 log.cc:1196] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/0c6512754f394ad2a6bca0bd276a76b8
I20240502 07:58:56.636025 25416 ts_tablet_manager.cc:1909] T 0c6512754f394ad2a6bca0bd276a76b8 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:56.636178 25418 ts_tablet_manager.cc:1909] T 0c6512754f394ad2a6bca0bd276a76b8 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:56.636641 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 0c6512754f394ad2a6bca0bd276a76b8 (table rebalance_test_table_2 [id=0937fa4568b0489ebd23bdd0b698a759]) successfully deleted
I20240502 07:58:56.636788 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 0c6512754f394ad2a6bca0bd276a76b8 (table rebalance_test_table_2 [id=0937fa4568b0489ebd23bdd0b698a759]) successfully deleted
I20240502 07:58:56.639261 22372 consensus_queue.cc:237] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:56.641335 22765 raft_consensus.cc:1261] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.641893 25528 consensus_queue.cc:1035] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.642226 22503 raft_consensus.cc:1261] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.643204 25528 consensus_queue.cc:1035] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.645387 24432 raft_consensus.cc:2932] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:56.645805 22503 raft_consensus.cc:2932] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
W20240502 07:58:56.648047 22305 consensus_peers.cc:485] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): Couldn't send request to peer 227eec4b5f014d349661f6e0b39b9055. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e6a499fd83554a2c8e3e837aedddb61b. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:56.647967 22765 raft_consensus.cc:2932] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:56.648764 22222 catalog_manager.cc:5484] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:56.669526 22765 consensus_queue.cc:237] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:56.671546 22372 raft_consensus.cc:1261] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.671550 22633 raft_consensus.cc:1261] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.672076 25504 consensus_queue.cc:1035] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.672292 24452 consensus_queue.cc:1035] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.674959 24452 raft_consensus.cc:2932] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
W20240502 07:58:56.676744 22701 consensus_peers.cc:485] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): Couldn't send request to peer 1aecef81733c47b3b468504934357231. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c5a30e1ebd7747eb911fba8437e42077. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:56.677026 22371 raft_consensus.cc:2932] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:56.677167 22633 raft_consensus.cc:2932] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:56.677712 22222 catalog_manager.cc:5484] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:56.698688 25403 consensus_queue.cc:1035] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.703059 25403 consensus_queue.cc:1035] T 9f916726456f4aa8936a9366468bc471 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.704264 22633 consensus_queue.cc:237] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:56.706115 22503 raft_consensus.cc:1261] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.706159 22371 raft_consensus.cc:1261] T ce0f9b48d2db4855a86eb20955fdc853 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.706475 24462 consensus_queue.cc:1035] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.706717 25256 consensus_queue.cc:1035] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.711230 24462 raft_consensus.cc:2932] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
W20240502 07:58:56.711863 22570 consensus_peers.cc:485] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): Couldn't send request to peer 43930dee0cd5498fa8e357a34dd71432. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ce0f9b48d2db4855a86eb20955fdc853. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:56.713222 22503 raft_consensus.cc:2932] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:56.713291 22371 raft_consensus.cc:2932] T ce0f9b48d2db4855a86eb20955fdc853 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:56.716620 22226 catalog_manager.cc:5484] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c reported cstate change: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:56.721752 22222 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:58:56.727245 25513 tablet.cc:1722] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:58:56.727561 25513 tablet.cc:1620] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
I20240502 07:58:56.727898 25353 tablet.cc:1722] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:58:56.728181 25353 tablet.cc:1620] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:58:56.728266 25520 tablet.cc:1722] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:58:56.728528 25520 tablet.cc:1620] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c: MemRowSet was empty: no flush needed.
I20240502 07:58:56.733660 22221 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=4fe10cc726b74369a67b32764fc4261b] alter complete (version 2)
I20240502 07:58:56.744240 22633 consensus_queue.cc:237] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.745641 22503 raft_consensus.cc:1261] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.745645 22371 raft_consensus.cc:1261] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.746016 24462 consensus_queue.cc:1035] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:58:56.746290 22569 consensus_peers.cc:485] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 257f0f4dca624c859d3b06cfd4246536. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:56.746326 25256 consensus_queue.cc:1035] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.748065 24462 raft_consensus.cc:2932] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:56.748494 22371 raft_consensus.cc:2932] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:56.751075 22222 catalog_manager.cc:5484] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:56.753801 22503 raft_consensus.cc:2932] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:56.777086 22503 consensus_queue.cc:237] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:56.778807 22765 raft_consensus.cc:1261] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.779036 22371 raft_consensus.cc:1261] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.779428 24696 consensus_queue.cc:1035] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.779645 25612 consensus_queue.cc:1035] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.781710 25612 raft_consensus.cc:2932] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
W20240502 07:58:56.783658 22436 consensus_peers.cc:485] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c -> Peer 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): Couldn't send request to peer 227eec4b5f014d349661f6e0b39b9055. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d4c1fadf79ab475b83e3303918f96d46. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:56.784154 22371 raft_consensus.cc:2932] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:56.784325 22765 raft_consensus.cc:2932] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:56.787331 22221 catalog_manager.cc:5484] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c reported cstate change: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:56.806823 22371 consensus_queue.cc:237] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:56.808607 22503 raft_consensus.cc:1261] T 02449bcab2294956918d1889485cfd5f P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.809266 22633 raft_consensus.cc:1261] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.809415 24432 consensus_queue.cc:1035] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.810293 24432 consensus_queue.cc:1035] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.812230 25527 raft_consensus.cc:2932] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
W20240502 07:58:56.812443 22308 consensus_peers.cc:485] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): Couldn't send request to peer 1aecef81733c47b3b468504934357231. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 02449bcab2294956918d1889485cfd5f. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:56.812705 22503 raft_consensus.cc:2932] T 02449bcab2294956918d1889485cfd5f P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:56.814939 22633 raft_consensus.cc:2932] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:56.815394 22226 catalog_manager.cc:5484] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:56.834125 22633 consensus_queue.cc:237] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:56.835472 22503 raft_consensus.cc:1261] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.835507 22765 raft_consensus.cc:1261] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.835928 25574 consensus_queue.cc:1035] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.836125 25587 consensus_queue.cc:1035] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:58:56.836373 22570 consensus_peers.cc:485] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): Couldn't send request to peer 43930dee0cd5498fa8e357a34dd71432. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 72964b1c935441ca840a46ff79921cc3. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:56.837776 25574 raft_consensus.cc:2932] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:56.837874 22222 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:58:56.839255 22503 raft_consensus.cc:2932] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:56.840981 22226 catalog_manager.cc:5484] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:56.841902 22765 raft_consensus.cc:2932] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:56.850714 25513 tablet.cc:1722] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:58:56.851101 25520 tablet.cc:1722] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:58:56.853075 25353 tablet.cc:1722] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:58:56.857038 22221 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=4fe10cc726b74369a67b32764fc4261b] alter complete (version 3)
I20240502 07:58:56.865702 25287 consensus_queue.cc:1035] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.870283 25287 consensus_queue.cc:1035] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.873126 22765 consensus_queue.cc:237] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:56.874807 22633 raft_consensus.cc:1261] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.874795 22371 raft_consensus.cc:1261] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.875223 24452 consensus_queue.cc:1035] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.875409 25548 consensus_queue.cc:1035] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.877298 24452 raft_consensus.cc:2932] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:56.877717 22633 raft_consensus.cc:2932] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
W20240502 07:58:56.880066 22700 consensus_peers.cc:485] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6dfeeb6b16a4f92bc756bf8933c7fb1. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:56.880887 22371 raft_consensus.cc:2932] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:56.880985 22222 catalog_manager.cc:5484] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:56.901705 22633 consensus_queue.cc:237] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:56.903136 22371 raft_consensus.cc:1261] T d9c0c75053be41f59786f1cf4fb3062c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.903333 22765 raft_consensus.cc:1261] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.903635 25574 consensus_queue.cc:1035] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.903785 25256 consensus_queue.cc:1035] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:58:56.906780 22567 consensus_peers.cc:485] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f -> Peer 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): Couldn't send request to peer 227eec4b5f014d349661f6e0b39b9055. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d9c0c75053be41f59786f1cf4fb3062c. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:56.910168 25256 raft_consensus.cc:2932] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:56.910550 22371 raft_consensus.cc:2932] T d9c0c75053be41f59786f1cf4fb3062c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:56.913234 22227 catalog_manager.cc:5484] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:56.914268 22765 raft_consensus.cc:2932] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:56.936724 22764 consensus_queue.cc:237] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:56.938221 22633 raft_consensus.cc:1261] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.938249 22503 raft_consensus.cc:1261] T 4e202e9cf0154eb9814a7e3ad93f1e42 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:58:56.938297 22701 consensus_peers.cc:485] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): Couldn't send request to peer 1aecef81733c47b3b468504934357231. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4e202e9cf0154eb9814a7e3ad93f1e42. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:56.938585 25548 consensus_queue.cc:1035] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.938776 24452 consensus_queue.cc:1035] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.940665 24452 raft_consensus.cc:2932] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:56.941133 22503 raft_consensus.cc:2932] T 4e202e9cf0154eb9814a7e3ad93f1e42 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:56.942755 22633 raft_consensus.cc:2932] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:56.944110 22221 catalog_manager.cc:5484] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:56.947649 25429 consensus_queue.cc:1035] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.952335 25429 consensus_queue.cc:1035] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:56.960673 22227 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:58:56.960831 22227 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:58:56.963105 22227 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 1478893ea6d54d5b80e19396196a3dd3
I20240502 07:58:56.963614 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet 1478893ea6d54d5b80e19396196a3dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:58:56.963737 22482 tablet_service.cc:1514] Processing DeleteTablet for tablet 1478893ea6d54d5b80e19396196a3dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:56.963747 24800 tablet_service.cc:1514] Processing DeleteTablet for tablet 1478893ea6d54d5b80e19396196a3dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:56 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:58:56.963928 25417 tablet_replica.cc:330] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:56.963989 25419 tablet_replica.cc:330] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:58:56.964066 25417 raft_consensus.cc:2229] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:56.964107 25419 raft_consensus.cc:2229] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:56.964183 25417 raft_consensus.cc:2258] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.964296 25419 raft_consensus.cc:2258] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.964709 25724 tablet_replica.cc:330] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:58:56.964848 25724 raft_consensus.cc:2229] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:56.964877 25419 ts_tablet_manager.cc:1875] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.964967 25724 raft_consensus.cc:2258] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:56.965044 25417 ts_tablet_manager.cc:1875] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.965533 25724 ts_tablet_manager.cc:1875] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:56.966822 25419 ts_tablet_manager.cc:1888] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:56.966879 25419 log.cc:1196] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/1478893ea6d54d5b80e19396196a3dd3
I20240502 07:58:56.967067 25419 ts_tablet_manager.cc:1909] T 1478893ea6d54d5b80e19396196a3dd3 P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:58:56.967356 25417 ts_tablet_manager.cc:1888] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:56.967439 25417 log.cc:1196] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/1478893ea6d54d5b80e19396196a3dd3
I20240502 07:58:56.967707 25417 ts_tablet_manager.cc:1909] T 1478893ea6d54d5b80e19396196a3dd3 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:58:56.967711 25724 ts_tablet_manager.cc:1888] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:56.967778 25724 log.cc:1196] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/1478893ea6d54d5b80e19396196a3dd3
I20240502 07:58:56.968004 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 1478893ea6d54d5b80e19396196a3dd3 (table rebalancer_dynamic_table_new_name [id=4fe10cc726b74369a67b32764fc4261b]) successfully deleted
I20240502 07:58:56.968061 25724 ts_tablet_manager.cc:1909] T 1478893ea6d54d5b80e19396196a3dd3 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:58:56.968374 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 1478893ea6d54d5b80e19396196a3dd3 (table rebalancer_dynamic_table_new_name [id=4fe10cc726b74369a67b32764fc4261b]) successfully deleted
I20240502 07:58:56.968652 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet 1478893ea6d54d5b80e19396196a3dd3 (table rebalancer_dynamic_table_new_name [id=4fe10cc726b74369a67b32764fc4261b]) successfully deleted
I20240502 07:58:56.973526 22633 consensus_queue.cc:237] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
W20240502 07:58:56.975003 22569 consensus_peers.cc:485] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 648a25ebebe44061bca2844ec5b048c5. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:56.975036 22764 raft_consensus.cc:1261] T 648a25ebebe44061bca2844ec5b048c5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.975212 22503 raft_consensus.cc:1261] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:56.975382 25587 consensus_queue.cc:1035] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.975672 25574 consensus_queue.cc:1035] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:56.977576 25574 raft_consensus.cc:2932] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:56.978045 22503 raft_consensus.cc:2932] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:56.979497 22764 raft_consensus.cc:2932] T 648a25ebebe44061bca2844ec5b048c5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:56.980440 22227 catalog_manager.cc:5484] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:56.999699 22633 consensus_queue.cc:237] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.001049 22371 raft_consensus.cc:1261] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:57.001049 22503 raft_consensus.cc:1261] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:58:57.001519 22570 consensus_peers.cc:485] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): Couldn't send request to peer 43930dee0cd5498fa8e357a34dd71432. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c9b0511ea0124d84ac5d3996bd1efcd2. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:57.001613 25574 consensus_queue.cc:1035] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:57.001957 25587 consensus_queue.cc:1035] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:57.003732 24462 raft_consensus.cc:2932] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:57.004168 22503 raft_consensus.cc:2932] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:57.005736 22371 raft_consensus.cc:2932] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:57.007877 22222 catalog_manager.cc:5484] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c reported cstate change: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:57.014779 22227 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:57.020915 24932 tablet_service.cc:1467] Processing CreateTablet for tablet c651077242584f57a165073de3e4a088 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=5b6a061be2624cb1852eb43e3a4cb66b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.021220 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c651077242584f57a165073de3e4a088. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.021164 25195 tablet_service.cc:1467] Processing CreateTablet for tablet c651077242584f57a165073de3e4a088 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=5b6a061be2624cb1852eb43e3a4cb66b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.021485 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c651077242584f57a165073de3e4a088. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.021757 22482 tablet_service.cc:1467] Processing CreateTablet for tablet cc97164c8cfe4f91afabcc606d9821ef (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=5b6a061be2624cb1852eb43e3a4cb66b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.022014 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc97164c8cfe4f91afabcc606d9821ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.022146 25194 tablet_service.cc:1467] Processing CreateTablet for tablet cc97164c8cfe4f91afabcc606d9821ef (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=5b6a061be2624cb1852eb43e3a4cb66b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.022369 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc97164c8cfe4f91afabcc606d9821ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.023882 25064 tablet_service.cc:1467] Processing CreateTablet for tablet c651077242584f57a165073de3e4a088 (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=5b6a061be2624cb1852eb43e3a4cb66b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.024147 25064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c651077242584f57a165073de3e4a088. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.024235 25763 tablet_bootstrap.cc:492] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:58:57.024677 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 4840fa82fcb241f391c4890fb735fe9a (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=5b6a061be2624cb1852eb43e3a4cb66b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.024890 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4840fa82fcb241f391c4890fb735fe9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.025092 25763 tablet_bootstrap.cc:654] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.026094 25763 tablet_bootstrap.cc:492] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:58:57.026165 25763 ts_tablet_manager.cc:1367] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:57.026471 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 4840fa82fcb241f391c4890fb735fe9a (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=5b6a061be2624cb1852eb43e3a4cb66b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.026556 25763 raft_consensus.cc:348] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.026649 25763 raft_consensus.cc:374] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.026676 25763 raft_consensus.cc:729] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.026696 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4840fa82fcb241f391c4890fb735fe9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.026775 25763 consensus_queue.cc:260] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.027034 25763 ts_tablet_manager.cc:1398] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.028118 25762 tablet_bootstrap.cc:492] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:58:57.029098 22351 tablet_service.cc:1467] Processing CreateTablet for tablet 4840fa82fcb241f391c4890fb735fe9a (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=5b6a061be2624cb1852eb43e3a4cb66b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.029388 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4840fa82fcb241f391c4890fb735fe9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.029902 25762 tablet_bootstrap.cc:654] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.030889 25762 tablet_bootstrap.cc:492] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:58:57.030973 25762 ts_tablet_manager.cc:1367] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:57.031404 25762 raft_consensus.cc:348] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.031530 25762 raft_consensus.cc:374] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.031565 25762 raft_consensus.cc:729] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.031695 25762 consensus_queue.cc:260] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.031827 22352 tablet_service.cc:1467] Processing CreateTablet for tablet cc97164c8cfe4f91afabcc606d9821ef (DEFAULT_TABLE table=rebalancer_extra_table_2 [id=5b6a061be2624cb1852eb43e3a4cb66b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.031970 25762 ts_tablet_manager.cc:1398] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.032058 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc97164c8cfe4f91afabcc606d9821ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.032138 25762 tablet_bootstrap.cc:492] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:58:57.032917 25762 tablet_bootstrap.cc:654] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.033030 25765 tablet_bootstrap.cc:492] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:58:57.033921 25765 tablet_bootstrap.cc:654] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.034502 25761 tablet_bootstrap.cc:492] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:58:57.035507 25761 tablet_bootstrap.cc:654] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.036478 25761 tablet_bootstrap.cc:492] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:58:57.036468 25762 tablet_bootstrap.cc:492] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:58:57.036559 25762 ts_tablet_manager.cc:1367] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:57.036566 25761 ts_tablet_manager.cc:1367] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:57.036980 25762 raft_consensus.cc:348] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.037086 25762 raft_consensus.cc:374] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.037122 25762 raft_consensus.cc:729] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.037251 25762 consensus_queue.cc:260] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.037598 25762 ts_tablet_manager.cc:1398] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.037801 25761 raft_consensus.cc:348] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.037963 25761 raft_consensus.cc:374] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.038038 25761 raft_consensus.cc:729] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.038235 25761 consensus_queue.cc:260] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.039116 25761 ts_tablet_manager.cc:1398] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:57.039359 25767 tablet_bootstrap.cc:492] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:57.039409 25761 tablet_bootstrap.cc:492] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:58:57.040678 25767 tablet_bootstrap.cc:654] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.040676 25761 tablet_bootstrap.cc:654] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.042032 25761 tablet_bootstrap.cc:492] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:58:57.042132 25761 ts_tablet_manager.cc:1367] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:57.042608 25761 raft_consensus.cc:348] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:57.042719 25761 raft_consensus.cc:374] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.042797 25761 raft_consensus.cc:729] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.042934 25761 consensus_queue.cc:260] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:57.043247 25761 ts_tablet_manager.cc:1398] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.043915 25765 tablet_bootstrap.cc:492] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:58:57.044019 25765 ts_tablet_manager.cc:1367] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20240502 07:58:57.044462 25765 raft_consensus.cc:348] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.044569 25765 raft_consensus.cc:374] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.044610 25765 raft_consensus.cc:729] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.044785 25765 consensus_queue.cc:260] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.045135 25765 ts_tablet_manager.cc:1398] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.045357 25765 tablet_bootstrap.cc:492] T 4840fa82fcb241f391c4890fb735fe9a P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:58:57.046294 25765 tablet_bootstrap.cc:654] T 4840fa82fcb241f391c4890fb735fe9a P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.048002 25767 tablet_bootstrap.cc:492] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:57.048108 25767 ts_tablet_manager.cc:1367] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 07:58:57.048506 25767 raft_consensus.cc:348] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:57.048604 25767 raft_consensus.cc:374] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.048636 25767 raft_consensus.cc:729] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.048770 25767 consensus_queue.cc:260] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:57.049005 25767 ts_tablet_manager.cc:1398] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:57.049162 25767 tablet_bootstrap.cc:492] T cc97164c8cfe4f91afabcc606d9821ef P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:57.050195 25767 tablet_bootstrap.cc:654] T cc97164c8cfe4f91afabcc606d9821ef P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.050488 25765 tablet_bootstrap.cc:492] T 4840fa82fcb241f391c4890fb735fe9a P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:58:57.050478 22764 consensus_queue.cc:237] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:57.050578 25765 ts_tablet_manager.cc:1367] T 4840fa82fcb241f391c4890fb735fe9a P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:57.051028 25765 raft_consensus.cc:348] T 4840fa82fcb241f391c4890fb735fe9a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:57.051138 25765 raft_consensus.cc:374] T 4840fa82fcb241f391c4890fb735fe9a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.051170 25765 raft_consensus.cc:729] T 4840fa82fcb241f391c4890fb735fe9a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.051281 25765 consensus_queue.cc:260] T 4840fa82fcb241f391c4890fb735fe9a P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:57.051533 25765 ts_tablet_manager.cc:1398] T 4840fa82fcb241f391c4890fb735fe9a P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.052564 22633 raft_consensus.cc:1261] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:57.052894 22371 raft_consensus.cc:1261] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:57.053236 25767 tablet_bootstrap.cc:492] T cc97164c8cfe4f91afabcc606d9821ef P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:57.053336 25767 ts_tablet_manager.cc:1367] T cc97164c8cfe4f91afabcc606d9821ef P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:57.053922 25767 raft_consensus.cc:348] T cc97164c8cfe4f91afabcc606d9821ef P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.054009 25767 raft_consensus.cc:374] T cc97164c8cfe4f91afabcc606d9821ef P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.054035 25767 raft_consensus.cc:729] T cc97164c8cfe4f91afabcc606d9821ef P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.054010 25548 consensus_queue.cc:1035] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:57.054144 25767 consensus_queue.cc:260] T cc97164c8cfe4f91afabcc606d9821ef P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.054322 25548 consensus_queue.cc:1035] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:57.054412 25767 ts_tablet_manager.cc:1398] T cc97164c8cfe4f91afabcc606d9821ef P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20240502 07:58:57.056181 22698 consensus_peers.cc:485] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): Couldn't send request to peer 227eec4b5f014d349661f6e0b39b9055. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fa14f8f327cb4221aabeedac16d258b6. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:57.058665 25548 raft_consensus.cc:2932] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:57.059075 22371 raft_consensus.cc:2932] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:57.060585 22633 raft_consensus.cc:2932] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:57.062125 22221 catalog_manager.cc:5484] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:57.065711 22222 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:58:57.071322 25194 tablet_service.cc:1467] Processing CreateTablet for tablet 140adf1ca68c4dfc9b3b713ecc7b582c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=49bcb615327b4dbf9ac425599e1cb97b]), partition=
I20240502 07:58:57.071427 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 140adf1ca68c4dfc9b3b713ecc7b582c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=49bcb615327b4dbf9ac425599e1cb97b]), partition=
I20240502 07:58:57.071588 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 140adf1ca68c4dfc9b3b713ecc7b582c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.071708 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 140adf1ca68c4dfc9b3b713ecc7b582c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.073274 25762 tablet_bootstrap.cc:492] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:58:57.073813 22612 tablet_service.cc:1467] Processing CreateTablet for tablet 140adf1ca68c4dfc9b3b713ecc7b582c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=49bcb615327b4dbf9ac425599e1cb97b]), partition=
I20240502 07:58:57.074080 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 140adf1ca68c4dfc9b3b713ecc7b582c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.074275 25762 tablet_bootstrap.cc:654] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.075412 25762 tablet_bootstrap.cc:492] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:58:57.075492 25762 ts_tablet_manager.cc:1367] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:57.075958 25762 raft_consensus.cc:348] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:57.076078 25762 raft_consensus.cc:374] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.076122 25762 raft_consensus.cc:729] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.076298 25790 tablet_bootstrap.cc:492] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:57.076277 25762 consensus_queue.cc:260] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:57.076668 25762 ts_tablet_manager.cc:1398] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:57.077402 25790 tablet_bootstrap.cc:654] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.078426 25788 tablet_bootstrap.cc:492] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:57.079506 25788 tablet_bootstrap.cc:654] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.080799 25788 tablet_bootstrap.cc:492] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:57.080906 25788 ts_tablet_manager.cc:1367] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:57.081300 25788 raft_consensus.cc:348] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:57.081455 25788 raft_consensus.cc:374] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.081521 25788 raft_consensus.cc:729] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.081714 25788 consensus_queue.cc:260] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:57.081960 25790 tablet_bootstrap.cc:492] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:57.082077 25788 ts_tablet_manager.cc:1398] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:57.082118 25790 ts_tablet_manager.cc:1367] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240502 07:58:57.082568 25790 raft_consensus.cc:348] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:57.082728 25790 raft_consensus.cc:374] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.082775 25790 raft_consensus.cc:729] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.082880 25790 consensus_queue.cc:260] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:57.083199 25790 ts_tablet_manager.cc:1398] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:57.087544 25486 raft_consensus.cc:482] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:57.087659 25486 raft_consensus.cc:504] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.088133 25486 leader_election.cc:288] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:58:57.088526 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c651077242584f57a165073de3e4a088" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:58:57.088670 25215 raft_consensus.cc:2454] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:58:57.088955 24888 leader_election.cc:302] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:58:57.089110 25486 raft_consensus.cc:2781] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:57.089159 25486 raft_consensus.cc:482] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:57.089207 25486 raft_consensus.cc:3037] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.090224 25486 raft_consensus.cc:504] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.090523 25486 leader_election.cc:288] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:58:57.091038 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c651077242584f57a165073de3e4a088" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:58:57.091142 25215 raft_consensus.cc:3037] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.091892 25215 raft_consensus.cc:2454] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:58:57.092793 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c651077242584f57a165073de3e4a088" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:58:57.092945 25084 raft_consensus.cc:2454] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:58:57.093066 24888 leader_election.cc:302] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:58:57.092808 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c651077242584f57a165073de3e4a088" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:58:57.093254 25083 raft_consensus.cc:3037] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.093668 25486 raft_consensus.cc:2781] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:57.093760 25486 raft_consensus.cc:686] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:58:57.093878 25486 consensus_queue.cc:237] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.094153 25083 raft_consensus.cc:2454] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:58:57.095602 22227 catalog_manager.cc:5484] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:57.106717 25486 raft_consensus.cc:482] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:57.106825 25486 raft_consensus.cc:504] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:57.107245 25486 leader_election.cc:288] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:57.107669 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4840fa82fcb241f391c4890fb735fe9a" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:58:57.107816 25083 raft_consensus.cc:2454] T 4840fa82fcb241f391c4890fb735fe9a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:58:57.108261 24885 leader_election.cc:302] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
I20240502 07:58:57.108477 25486 raft_consensus.cc:2781] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:57.108534 25486 raft_consensus.cc:482] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:57.108574 25486 raft_consensus.cc:3037] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.111177 25486 raft_consensus.cc:504] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:57.111752 25486 leader_election.cc:288] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:57.111826 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4840fa82fcb241f391c4890fb735fe9a" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:58:57.111955 25083 raft_consensus.cc:3037] T 4840fa82fcb241f391c4890fb735fe9a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.112329 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4840fa82fcb241f391c4890fb735fe9a" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:57.112300 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4840fa82fcb241f391c4890fb735fe9a" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:57.112445 22372 raft_consensus.cc:3037] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.112933 25083 raft_consensus.cc:2454] T 4840fa82fcb241f391c4890fb735fe9a P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:58:57.113412 22372 raft_consensus.cc:2454] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:58:57.113634 24885 leader_election.cc:302] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
I20240502 07:58:57.113793 25486 raft_consensus.cc:2781] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:57.113890 25486 raft_consensus.cc:686] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:58:57.114020 25486 consensus_queue.cc:237] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:57.115520 22227 catalog_manager.cc:5484] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:58:57.118902 22372 consensus_queue.cc:237] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:57.120983 22633 raft_consensus.cc:1261] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:57.121112 22764 raft_consensus.cc:1261] T c2422c666baa48078030c43d1c0938f9 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:57.121706 24432 consensus_queue.cc:1035] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:57.122028 24432 consensus_queue.cc:1035] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:58:57.124554 22307 consensus_peers.cc:485] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c2422c666baa48078030c43d1c0938f9. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:57.124712 25528 raft_consensus.cc:2932] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:57.125780 22764 raft_consensus.cc:2932] T c2422c666baa48078030c43d1c0938f9 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:57.125962 22227 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_4" } modify_external_catalogs: true
I20240502 07:58:57.126077 22227 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_4" } modify_external_catalogs: true
I20240502 07:58:57.127077 22633 raft_consensus.cc:2932] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:57.129081 22222 catalog_manager.cc:5484] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:57.131263 22227 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 1ec8ca8bbe2341dc8000de45ef289415
I20240502 07:58:57.131837 22352 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ec8ca8bbe2341dc8000de45ef289415 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:57.131956 22227 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 58c28df209214cb6a15d41423a5cbd32
I20240502 07:58:57.132139 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ec8ca8bbe2341dc8000de45ef289415 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:58:57.132159 22482 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ec8ca8bbe2341dc8000de45ef289415 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:57.132321 25417 tablet_replica.cc:330] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:57.132323 25416 tablet_replica.cc:330] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:57.132447 25416 raft_consensus.cc:2229] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:57.132462 25417 raft_consensus.cc:2229] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:57.132577 25417 raft_consensus.cc:2258] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.132575 25416 raft_consensus.cc:2258] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.132784 22227 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 696f4ce96be64070a1aa2760df43720b
I20240502 07:58:57.133059 25416 ts_tablet_manager.cc:1875] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.133150 25417 ts_tablet_manager.cc:1875] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.133386 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet 696f4ce96be64070a1aa2760df43720b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:57.133553 22482 tablet_service.cc:1514] Processing DeleteTablet for tablet 58c28df209214cb6a15d41423a5cbd32 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:57.133698 22352 tablet_service.cc:1514] Processing DeleteTablet for tablet 58c28df209214cb6a15d41423a5cbd32 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:57.135183 25416 ts_tablet_manager.cc:1888] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:57.135183 25417 ts_tablet_manager.cc:1888] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:57.135248 25416 log.cc:1196] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/1ec8ca8bbe2341dc8000de45ef289415
I20240502 07:58:57.135248 25417 log.cc:1196] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/1ec8ca8bbe2341dc8000de45ef289415
I20240502 07:58:57.135516 25416 ts_tablet_manager.cc:1909] T 1ec8ca8bbe2341dc8000de45ef289415 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:57.135514 25417 ts_tablet_manager.cc:1909] T 1ec8ca8bbe2341dc8000de45ef289415 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:58:57.135712 25419 tablet_replica.cc:330] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:58:57.135856 25419 raft_consensus.cc:2229] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:57.136023 25419 raft_consensus.cc:2258] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.136276 25416 tablet_replica.cc:330] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:57.136360 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 1ec8ca8bbe2341dc8000de45ef289415 (table rebalance_test_table_4 [id=74c99be4bec647c09e6275ef44e45334]) successfully deleted
I20240502 07:58:57.136404 25416 raft_consensus.cc:2229] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:57.136269 25417 tablet_replica.cc:330] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:57.136477 25419 ts_tablet_manager.cc:1875] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.136493 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 1ec8ca8bbe2341dc8000de45ef289415 (table rebalance_test_table_4 [id=74c99be4bec647c09e6275ef44e45334]) successfully deleted
I20240502 07:58:57.136518 25416 raft_consensus.cc:2258] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.136601 25417 raft_consensus.cc:2229] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:57.136736 25417 raft_consensus.cc:2258] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.137185 25417 ts_tablet_manager.cc:1875] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.137391 25416 ts_tablet_manager.cc:1875] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.138139 25419 ts_tablet_manager.cc:1888] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:57.138204 25419 log.cc:1196] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/1ec8ca8bbe2341dc8000de45ef289415
I20240502 07:58:57.138434 25419 ts_tablet_manager.cc:1909] T 1ec8ca8bbe2341dc8000de45ef289415 P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:58:57.139161 25417 ts_tablet_manager.cc:1888] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:57.139221 25417 log.cc:1196] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/58c28df209214cb6a15d41423a5cbd32
I20240502 07:58:57.139206 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet 58c28df209214cb6a15d41423a5cbd32 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:57.139286 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 1ec8ca8bbe2341dc8000de45ef289415 (table rebalance_test_table_4 [id=74c99be4bec647c09e6275ef44e45334]) successfully deleted
I20240502 07:58:57.139446 22744 tablet_service.cc:1514] Processing DeleteTablet for tablet 696f4ce96be64070a1aa2760df43720b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:57.139477 25417 ts_tablet_manager.cc:1909] T 58c28df209214cb6a15d41423a5cbd32 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:58:57.139813 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet 696f4ce96be64070a1aa2760df43720b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:58:57.139761 25416 ts_tablet_manager.cc:1888] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:57.140141 25416 log.cc:1196] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/696f4ce96be64070a1aa2760df43720b
I20240502 07:58:57.140136 25419 tablet_replica.cc:330] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:58:57.140177 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 58c28df209214cb6a15d41423a5cbd32 (table rebalance_test_table_4 [id=74c99be4bec647c09e6275ef44e45334]) successfully deleted
I20240502 07:58:57.140265 25419 raft_consensus.cc:2229] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:57.140396 25419 raft_consensus.cc:2258] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.140506 25416 ts_tablet_manager.cc:1909] T 696f4ce96be64070a1aa2760df43720b P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:57.140858 25419 ts_tablet_manager.cc:1875] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.141067 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 696f4ce96be64070a1aa2760df43720b (table rebalance_test_table_4 [id=74c99be4bec647c09e6275ef44e45334]) successfully deleted
I20240502 07:58:57.141636 25812 tablet_replica.cc:330] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:57.141942 25812 raft_consensus.cc:2229] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:57.142132 25812 raft_consensus.cc:2258] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.142506 25416 tablet_replica.cc:330] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:57.142700 25416 raft_consensus.cc:2229] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:57.142704 25812 ts_tablet_manager.cc:1875] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.142894 25416 raft_consensus.cc:2258] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.142973 25419 ts_tablet_manager.cc:1888] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:57.143043 25419 log.cc:1196] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/696f4ce96be64070a1aa2760df43720b
I20240502 07:58:57.143288 25419 ts_tablet_manager.cc:1909] T 696f4ce96be64070a1aa2760df43720b P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:58:57.143399 25416 ts_tablet_manager.cc:1875] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.144261 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 696f4ce96be64070a1aa2760df43720b (table rebalance_test_table_4 [id=74c99be4bec647c09e6275ef44e45334]) successfully deleted
I20240502 07:58:57.145671 25416 ts_tablet_manager.cc:1888] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:57.145737 25416 log.cc:1196] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/58c28df209214cb6a15d41423a5cbd32
I20240502 07:58:57.145996 25416 ts_tablet_manager.cc:1909] T 58c28df209214cb6a15d41423a5cbd32 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:57.146123 25812 ts_tablet_manager.cc:1888] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:57.146196 25812 log.cc:1196] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/58c28df209214cb6a15d41423a5cbd32
I20240502 07:58:57.146414 25812 ts_tablet_manager.cc:1909] T 58c28df209214cb6a15d41423a5cbd32 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:57.146690 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 58c28df209214cb6a15d41423a5cbd32 (table rebalance_test_table_4 [id=74c99be4bec647c09e6275ef44e45334]) successfully deleted
I20240502 07:58:57.147101 25812 tablet_replica.cc:330] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:57.147183 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 58c28df209214cb6a15d41423a5cbd32 (table rebalance_test_table_4 [id=74c99be4bec647c09e6275ef44e45334]) successfully deleted
I20240502 07:58:57.147239 25812 raft_consensus.cc:2229] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:57.147403 25812 raft_consensus.cc:2258] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.147843 25812 ts_tablet_manager.cc:1875] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.149809 25812 ts_tablet_manager.cc:1888] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:57.149904 25812 log.cc:1196] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/696f4ce96be64070a1aa2760df43720b
I20240502 07:58:57.150156 25812 ts_tablet_manager.cc:1909] T 696f4ce96be64070a1aa2760df43720b P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:57.150913 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 696f4ce96be64070a1aa2760df43720b (table rebalance_test_table_4 [id=74c99be4bec647c09e6275ef44e45334]) successfully deleted
I20240502 07:58:57.159482 25823 ts_tablet_manager.cc:902] T c5a30e1ebd7747eb911fba8437e42077 P 1aecef81733c47b3b468504934357231: Initiating tablet copy from peer f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:57.160604 25823 tablet_copy_client.cc:308] T c5a30e1ebd7747eb911fba8437e42077 P 1aecef81733c47b3b468504934357231: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.132:41673
I20240502 07:58:57.164868 22785 tablet_copy_service.cc:140] P f00f3a1876444ad2bf9c919068ab8014: Received BeginTabletCopySession request for tablet c5a30e1ebd7747eb911fba8437e42077 from peer 1aecef81733c47b3b468504934357231 ({username='slave'} at 127.15.129.134:51231)
I20240502 07:58:57.165004 22785 tablet_copy_service.cc:161] P f00f3a1876444ad2bf9c919068ab8014: Beginning new tablet copy session on tablet c5a30e1ebd7747eb911fba8437e42077 from peer 1aecef81733c47b3b468504934357231 at {username='slave'} at 127.15.129.134:51231: session id = 1aecef81733c47b3b468504934357231-c5a30e1ebd7747eb911fba8437e42077
I20240502 07:58:57.166527 22785 tablet_copy_source_session.cc:205] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.168215 25823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5a30e1ebd7747eb911fba8437e42077. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.168202 22372 consensus_queue.cc:237] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.169849 22503 raft_consensus.cc:1261] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:57.170440 22633 raft_consensus.cc:1261] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:57.170698 25823 tablet_copy_client.cc:783] T c5a30e1ebd7747eb911fba8437e42077 P 1aecef81733c47b3b468504934357231: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.170886 25823 tablet_copy_client.cc:647] T c5a30e1ebd7747eb911fba8437e42077 P 1aecef81733c47b3b468504934357231: tablet copy: Starting download of 1 WAL segments...
W20240502 07:58:57.171242 22308 consensus_peers.cc:485] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): Couldn't send request to peer 1aecef81733c47b3b468504934357231. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 45925e9713cb4b748efde06f77cf3702. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:57.171366 24432 consensus_queue.cc:1035] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:57.171521 25527 consensus_queue.cc:1035] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:57.173295 25823 tablet_copy_client.cc:523] T c5a30e1ebd7747eb911fba8437e42077 P 1aecef81733c47b3b468504934357231: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.174136 24432 raft_consensus.cc:2932] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:57.174580 22633 raft_consensus.cc:2932] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:57.175087 25823 tablet_bootstrap.cc:492] T c5a30e1ebd7747eb911fba8437e42077 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:58:57.175797 25429 raft_consensus.cc:482] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:57.175899 25429 raft_consensus.cc:504] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:57.176445 25429 leader_election.cc:288] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:57.179741 22227 catalog_manager.cc:5484] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:57.180951 22503 raft_consensus.cc:2932] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:57.183981 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "140adf1ca68c4dfc9b3b713ecc7b582c" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:57.184141 22764 raft_consensus.cc:2454] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:58:57.184445 25151 leader_election.cc:302] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:57.184649 25429 raft_consensus.cc:2781] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:57.184703 25429 raft_consensus.cc:482] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:57.184749 25429 raft_consensus.cc:3037] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.185830 25429 raft_consensus.cc:504] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:57.186250 25429 leader_election.cc:288] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:57.186833 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "140adf1ca68c4dfc9b3b713ecc7b582c" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:57.186964 22765 raft_consensus.cc:3037] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.189111 22765 raft_consensus.cc:2454] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:58:57.189890 25151 leader_election.cc:302] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:57.191623 22633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "140adf1ca68c4dfc9b3b713ecc7b582c" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:58:57.191783 25429 raft_consensus.cc:2781] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:57.191795 22633 raft_consensus.cc:2454] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:58:57.191879 25429 raft_consensus.cc:686] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:58:57.192013 25429 consensus_queue.cc:237] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:58:57.191653 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "140adf1ca68c4dfc9b3b713ecc7b582c" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:58:57.193159 22634 raft_consensus.cc:3037] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.193879 22227 catalog_manager.cc:5484] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:58:57.194216 22634 raft_consensus.cc:2454] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:58:57.206526 25856 ts_tablet_manager.cc:902] T ce0f9b48d2db4855a86eb20955fdc853 P 43930dee0cd5498fa8e357a34dd71432: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:57.206905 22764 consensus_queue.cc:237] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.207543 25856 tablet_copy_client.cc:308] T ce0f9b48d2db4855a86eb20955fdc853 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:58:57.208992 22654 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet ce0f9b48d2db4855a86eb20955fdc853 from peer 43930dee0cd5498fa8e357a34dd71432 ({username='slave'} at 127.15.129.136:34023)
I20240502 07:58:57.209053 22503 raft_consensus.cc:1261] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:57.209092 22372 raft_consensus.cc:1261] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:57.209146 22654 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet ce0f9b48d2db4855a86eb20955fdc853 from peer 43930dee0cd5498fa8e357a34dd71432 at {username='slave'} at 127.15.129.136:34023: session id = 43930dee0cd5498fa8e357a34dd71432-ce0f9b48d2db4855a86eb20955fdc853
I20240502 07:58:57.211109 25504 consensus_queue.cc:1035] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:57.211656 25504 consensus_queue.cc:1035] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:57.212924 22654 tablet_copy_source_session.cc:205] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.215490 25823 tablet_bootstrap.cc:492] T c5a30e1ebd7747eb911fba8437e42077 P 1aecef81733c47b3b468504934357231: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.215771 25856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce0f9b48d2db4855a86eb20955fdc853. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.215866 25823 tablet_bootstrap.cc:492] T c5a30e1ebd7747eb911fba8437e42077 P 1aecef81733c47b3b468504934357231: Bootstrap complete.
I20240502 07:58:57.216049 25823 ts_tablet_manager.cc:1367] T c5a30e1ebd7747eb911fba8437e42077 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.041s user 0.023s sys 0.005s
W20240502 07:58:57.216087 22701 consensus_peers.cc:485] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 -> Peer 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): Couldn't send request to peer 43930dee0cd5498fa8e357a34dd71432. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b9557af8565d4588b119f1915e76c3dc. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:57.216511 25823 raft_consensus.cc:348] T c5a30e1ebd7747eb911fba8437e42077 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.216639 25823 raft_consensus.cc:729] T c5a30e1ebd7747eb911fba8437e42077 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.216786 25823 consensus_queue.cc:260] T c5a30e1ebd7747eb911fba8437e42077 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.217154 25823 ts_tablet_manager.cc:1398] T c5a30e1ebd7747eb911fba8437e42077 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 07:58:57.217794 22785 tablet_copy_service.cc:342] P f00f3a1876444ad2bf9c919068ab8014: Request end of tablet copy session 1aecef81733c47b3b468504934357231-c5a30e1ebd7747eb911fba8437e42077 received from {username='slave'} at 127.15.129.134:51231
I20240502 07:58:57.217885 22785 tablet_copy_service.cc:433] P f00f3a1876444ad2bf9c919068ab8014: ending tablet copy session 1aecef81733c47b3b468504934357231-c5a30e1ebd7747eb911fba8437e42077 on tablet c5a30e1ebd7747eb911fba8437e42077 with peer 1aecef81733c47b3b468504934357231
I20240502 07:58:57.218073 25504 raft_consensus.cc:2932] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:57.218453 22372 raft_consensus.cc:2932] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:57.218853 25864 ts_tablet_manager.cc:902] T 72964b1c935441ca840a46ff79921cc3 P 43930dee0cd5498fa8e357a34dd71432: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:57.220341 25823 ts_tablet_manager.cc:902] T 02449bcab2294956918d1889485cfd5f P 1aecef81733c47b3b468504934357231: Initiating tablet copy from peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:57.220783 25823 tablet_copy_client.cc:308] T 02449bcab2294956918d1889485cfd5f P 1aecef81733c47b3b468504934357231: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.129:38535
I20240502 07:58:57.220984 22503 raft_consensus.cc:2932] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:57.223093 22227 catalog_manager.cc:5484] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:57.223940 25864 tablet_copy_client.cc:308] T 72964b1c935441ca840a46ff79921cc3 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:58:57.224476 22654 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet 72964b1c935441ca840a46ff79921cc3 from peer 43930dee0cd5498fa8e357a34dd71432 ({username='slave'} at 127.15.129.136:34023)
I20240502 07:58:57.224591 22654 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet 72964b1c935441ca840a46ff79921cc3 from peer 43930dee0cd5498fa8e357a34dd71432 at {username='slave'} at 127.15.129.136:34023: session id = 43930dee0cd5498fa8e357a34dd71432-72964b1c935441ca840a46ff79921cc3
I20240502 07:58:57.225536 22654 tablet_copy_source_session.cc:205] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.225880 25856 tablet_copy_client.cc:783] T ce0f9b48d2db4855a86eb20955fdc853 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.226261 25856 tablet_copy_client.cc:647] T ce0f9b48d2db4855a86eb20955fdc853 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.226804 25864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72964b1c935441ca840a46ff79921cc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.228469 25856 tablet_copy_client.cc:523] T ce0f9b48d2db4855a86eb20955fdc853 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.229032 22392 tablet_copy_service.cc:140] P b533a7d8d97943609c6e28e3e36b78a4: Received BeginTabletCopySession request for tablet 02449bcab2294956918d1889485cfd5f from peer 1aecef81733c47b3b468504934357231 ({username='slave'} at 127.15.129.134:45925)
I20240502 07:58:57.229183 22392 tablet_copy_service.cc:161] P b533a7d8d97943609c6e28e3e36b78a4: Beginning new tablet copy session on tablet 02449bcab2294956918d1889485cfd5f from peer 1aecef81733c47b3b468504934357231 at {username='slave'} at 127.15.129.134:45925: session id = 1aecef81733c47b3b468504934357231-02449bcab2294956918d1889485cfd5f
I20240502 07:58:57.229422 25864 tablet_copy_client.cc:783] T 72964b1c935441ca840a46ff79921cc3 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.229598 25864 tablet_copy_client.cc:647] T 72964b1c935441ca840a46ff79921cc3 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.230603 25856 tablet_bootstrap.cc:492] T ce0f9b48d2db4855a86eb20955fdc853 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:58:57.230731 22392 tablet_copy_source_session.cc:205] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.231930 25823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02449bcab2294956918d1889485cfd5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.232807 25864 tablet_copy_client.cc:523] T 72964b1c935441ca840a46ff79921cc3 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.234179 25864 tablet_bootstrap.cc:492] T 72964b1c935441ca840a46ff79921cc3 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:58:57.234258 25823 tablet_copy_client.cc:783] T 02449bcab2294956918d1889485cfd5f P 1aecef81733c47b3b468504934357231: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.234419 25823 tablet_copy_client.cc:647] T 02449bcab2294956918d1889485cfd5f P 1aecef81733c47b3b468504934357231: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.236254 25823 tablet_copy_client.cc:523] T 02449bcab2294956918d1889485cfd5f P 1aecef81733c47b3b468504934357231: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.237927 25823 tablet_bootstrap.cc:492] T 02449bcab2294956918d1889485cfd5f P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:58:57.240098 25877 ts_tablet_manager.cc:902] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055: Initiating tablet copy from peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:57.246109 25877 tablet_copy_client.cc:308] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.129:38535
I20240502 07:58:57.252266 22392 tablet_copy_service.cc:140] P b533a7d8d97943609c6e28e3e36b78a4: Received BeginTabletCopySession request for tablet e6a499fd83554a2c8e3e837aedddb61b from peer 227eec4b5f014d349661f6e0b39b9055 ({username='slave'} at 127.15.129.135:41433)
I20240502 07:58:57.252393 22392 tablet_copy_service.cc:161] P b533a7d8d97943609c6e28e3e36b78a4: Beginning new tablet copy session on tablet e6a499fd83554a2c8e3e837aedddb61b from peer 227eec4b5f014d349661f6e0b39b9055 at {username='slave'} at 127.15.129.135:41433: session id = 227eec4b5f014d349661f6e0b39b9055-e6a499fd83554a2c8e3e837aedddb61b
I20240502 07:58:57.253443 22392 tablet_copy_source_session.cc:205] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.254902 25877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6a499fd83554a2c8e3e837aedddb61b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.257546 25877 tablet_copy_client.cc:783] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.257776 25877 tablet_copy_client.cc:647] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.259403 25877 tablet_copy_client.cc:523] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.261075 25877 tablet_bootstrap.cc:492] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:58:57.272719 25402 raft_consensus.cc:482] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:57.272857 25402 raft_consensus.cc:504] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.273401 25402 leader_election.cc:288] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:58:57.273985 25864 tablet_bootstrap.cc:492] T 72964b1c935441ca840a46ff79921cc3 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.274147 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc97164c8cfe4f91afabcc606d9821ef" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:57.274297 22372 raft_consensus.cc:2454] T cc97164c8cfe4f91afabcc606d9821ef P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:57.274327 25864 tablet_bootstrap.cc:492] T 72964b1c935441ca840a46ff79921cc3 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap complete.
I20240502 07:58:57.274621 25864 ts_tablet_manager.cc:1367] T 72964b1c935441ca840a46ff79921cc3 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.040s user 0.033s sys 0.002s
I20240502 07:58:57.274629 22439 leader_election.cc:302] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:57.274798 25402 raft_consensus.cc:2781] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:57.274852 25402 raft_consensus.cc:482] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:57.274899 25402 raft_consensus.cc:3037] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.275123 25864 raft_consensus.cc:348] T 72964b1c935441ca840a46ff79921cc3 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.275241 25864 raft_consensus.cc:729] T 72964b1c935441ca840a46ff79921cc3 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.275383 25864 consensus_queue.cc:260] T 72964b1c935441ca840a46ff79921cc3 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.275679 25864 ts_tablet_manager.cc:1398] T 72964b1c935441ca840a46ff79921cc3 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.275904 25402 raft_consensus.cc:504] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.276250 25402 leader_election.cc:288] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:58:57.277114 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc97164c8cfe4f91afabcc606d9821ef" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:57.277228 22372 raft_consensus.cc:3037] T cc97164c8cfe4f91afabcc606d9821ef P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.278069 22654 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session 43930dee0cd5498fa8e357a34dd71432-72964b1c935441ca840a46ff79921cc3 received from {username='slave'} at 127.15.129.136:34023
I20240502 07:58:57.278172 22654 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session 43930dee0cd5498fa8e357a34dd71432-72964b1c935441ca840a46ff79921cc3 on tablet 72964b1c935441ca840a46ff79921cc3 with peer 43930dee0cd5498fa8e357a34dd71432
I20240502 07:58:57.278303 22372 raft_consensus.cc:2454] T cc97164c8cfe4f91afabcc606d9821ef P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:58:57.278666 22439 leader_election.cc:302] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:58:57.278930 25402 raft_consensus.cc:2781] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:57.279032 25402 raft_consensus.cc:686] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:58:57.279111 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc97164c8cfe4f91afabcc606d9821ef" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:58:57.279170 25402 consensus_queue.cc:237] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.279215 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc97164c8cfe4f91afabcc606d9821ef" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:58:57.279244 25215 raft_consensus.cc:2454] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:58:57.282624 22227 catalog_manager.cc:5484] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:57.286823 25823 tablet_bootstrap.cc:492] T 02449bcab2294956918d1889485cfd5f P 1aecef81733c47b3b468504934357231: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.287137 25823 tablet_bootstrap.cc:492] T 02449bcab2294956918d1889485cfd5f P 1aecef81733c47b3b468504934357231: Bootstrap complete.
I20240502 07:58:57.287329 25823 ts_tablet_manager.cc:1367] T 02449bcab2294956918d1889485cfd5f P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.049s user 0.028s sys 0.000s
I20240502 07:58:57.287860 25823 raft_consensus.cc:348] T 02449bcab2294956918d1889485cfd5f P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.287979 25823 raft_consensus.cc:729] T 02449bcab2294956918d1889485cfd5f P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.288187 25823 consensus_queue.cc:260] T 02449bcab2294956918d1889485cfd5f P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.288585 25823 ts_tablet_manager.cc:1398] T 02449bcab2294956918d1889485cfd5f P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:58:57.289157 22392 tablet_copy_service.cc:342] P b533a7d8d97943609c6e28e3e36b78a4: Request end of tablet copy session 1aecef81733c47b3b468504934357231-02449bcab2294956918d1889485cfd5f received from {username='slave'} at 127.15.129.134:45925
I20240502 07:58:57.289289 22392 tablet_copy_service.cc:433] P b533a7d8d97943609c6e28e3e36b78a4: ending tablet copy session 1aecef81733c47b3b468504934357231-02449bcab2294956918d1889485cfd5f on tablet 02449bcab2294956918d1889485cfd5f with peer 1aecef81733c47b3b468504934357231
I20240502 07:58:57.293489 25856 tablet_bootstrap.cc:492] T ce0f9b48d2db4855a86eb20955fdc853 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.293784 25856 tablet_bootstrap.cc:492] T ce0f9b48d2db4855a86eb20955fdc853 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap complete.
I20240502 07:58:57.293957 25856 ts_tablet_manager.cc:1367] T ce0f9b48d2db4855a86eb20955fdc853 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.063s user 0.028s sys 0.002s
I20240502 07:58:57.294421 25856 raft_consensus.cc:348] T ce0f9b48d2db4855a86eb20955fdc853 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.294553 25856 raft_consensus.cc:729] T ce0f9b48d2db4855a86eb20955fdc853 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.294700 25856 consensus_queue.cc:260] T ce0f9b48d2db4855a86eb20955fdc853 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.295509 25856 ts_tablet_manager.cc:1398] T ce0f9b48d2db4855a86eb20955fdc853 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:57.296195 22654 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session 43930dee0cd5498fa8e357a34dd71432-ce0f9b48d2db4855a86eb20955fdc853 received from {username='slave'} at 127.15.129.136:34023
I20240502 07:58:57.296279 22654 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session 43930dee0cd5498fa8e357a34dd71432-ce0f9b48d2db4855a86eb20955fdc853 on tablet ce0f9b48d2db4855a86eb20955fdc853 with peer 43930dee0cd5498fa8e357a34dd71432
I20240502 07:58:57.300143 22372 consensus_queue.cc:237] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:57.301601 22764 raft_consensus.cc:1261] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:57.301728 22503 raft_consensus.cc:1261] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:58:57.301918 22307 consensus_peers.cc:485] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4dc697950c8140878b50550fe0ff458d. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:57.302109 25810 consensus_queue.cc:1035] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:57.302474 25810 consensus_queue.cc:1035] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:57.304581 25810 raft_consensus.cc:2932] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:57.305541 22764 raft_consensus.cc:2932] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:57.305804 25877 tablet_bootstrap.cc:492] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.306125 25877 tablet_bootstrap.cc:492] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055: Bootstrap complete.
I20240502 07:58:57.306288 25877 ts_tablet_manager.cc:1367] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.045s user 0.027s sys 0.005s
I20240502 07:58:57.306526 22503 raft_consensus.cc:2932] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:57.306792 25877 raft_consensus.cc:348] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:57.306901 25877 raft_consensus.cc:729] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.307056 25877 consensus_queue.cc:260] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:57.307344 25877 ts_tablet_manager.cc:1398] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.307895 22392 tablet_copy_service.cc:342] P b533a7d8d97943609c6e28e3e36b78a4: Request end of tablet copy session 227eec4b5f014d349661f6e0b39b9055-e6a499fd83554a2c8e3e837aedddb61b received from {username='slave'} at 127.15.129.135:41433
I20240502 07:58:57.307971 22392 tablet_copy_service.cc:433] P b533a7d8d97943609c6e28e3e36b78a4: ending tablet copy session 227eec4b5f014d349661f6e0b39b9055-e6a499fd83554a2c8e3e837aedddb61b on tablet e6a499fd83554a2c8e3e837aedddb61b with peer 227eec4b5f014d349661f6e0b39b9055
I20240502 07:58:57.309511 22226 catalog_manager.cc:5484] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:57.315497 25907 ts_tablet_manager.cc:902] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:57.315485 22221 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:58:57.316124 25907 tablet_copy_client.cc:308] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:58:57.320303 22634 raft_consensus.cc:1261] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:57.320881 22764 raft_consensus.cc:1261] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:57.320878 25429 consensus_queue.cc:1035] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.321414 25429 consensus_queue.cc:1035] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.321497 22654 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet 257f0f4dca624c859d3b06cfd4246536 from peer 4b4e7eeb7f3f42119c0363a79febc54b ({username='slave'} at 127.15.129.133:55755)
I20240502 07:58:57.321597 22654 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet 257f0f4dca624c859d3b06cfd4246536 from peer 4b4e7eeb7f3f42119c0363a79febc54b at {username='slave'} at 127.15.129.133:55755: session id = 4b4e7eeb7f3f42119c0363a79febc54b-257f0f4dca624c859d3b06cfd4246536
I20240502 07:58:57.322533 22654 tablet_copy_source_session.cc:205] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.324461 22221 catalog_manager.cc:5576] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673) does not have the latest schema for tablet 140adf1ca68c4dfc9b3b713ecc7b582c (table rebalancer_dynamic_table [id=49bcb615327b4dbf9ac425599e1cb97b]). Expected version 1 got 0
I20240502 07:58:57.325670 25907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 257f0f4dca624c859d3b06cfd4246536. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.326669 25513 tablet.cc:1722] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:58:57.326843 22227 catalog_manager.cc:5576] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703) does not have the latest schema for tablet 140adf1ca68c4dfc9b3b713ecc7b582c (table rebalancer_dynamic_table [id=49bcb615327b4dbf9ac425599e1cb97b]). Expected version 1 got 0
I20240502 07:58:57.326939 25670 mvcc.cc:205] Tried to move back new op lower bound from 7023152076059312128 to 7023152075540008960. Current Snapshot: MvccSnapshot[applied={T|T < 7023152076059312128}]
I20240502 07:58:57.326996 25513 tablet.cc:1620] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
I20240502 07:58:57.327217 25921 tablet.cc:1722] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:58:57.329922 22222 catalog_manager.cc:5576] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673) does not have the latest schema for tablet 140adf1ca68c4dfc9b3b713ecc7b582c (table rebalancer_dynamic_table [id=49bcb615327b4dbf9ac425599e1cb97b]). Expected version 1 got 0
I20240502 07:58:57.330421 25921 tablet.cc:1620] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:58:57.333513 22226 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=49bcb615327b4dbf9ac425599e1cb97b] alter complete (version 1)
I20240502 07:58:57.335093 25907 tablet_copy_client.cc:783] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.335279 25907 tablet_copy_client.cc:647] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.336127 25922 tablet.cc:1722] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:58:57.336882 25922 tablet.cc:1620] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:58:57.337509 25907 tablet_copy_client.cc:523] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.338560 22764 consensus_queue.cc:237] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:57.338989 25907 tablet_bootstrap.cc:492] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
W20240502 07:58:57.340312 22698 consensus_peers.cc:485] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 -> Peer 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): Couldn't send request to peer 227eec4b5f014d349661f6e0b39b9055. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 92162f8f5fcb4baf96fa8004b7b41b9b. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:57.340556 22503 raft_consensus.cc:1261] T 92162f8f5fcb4baf96fa8004b7b41b9b P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:57.340623 22634 raft_consensus.cc:1261] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:57.340941 25710 consensus_queue.cc:1035] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:57.341117 24452 consensus_queue.cc:1035] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:57.342995 25710 raft_consensus.cc:2932] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:57.345043 22634 raft_consensus.cc:2932] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:57.346086 22221 catalog_manager.cc:5484] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:57.347024 22503 raft_consensus.cc:2932] T 92162f8f5fcb4baf96fa8004b7b41b9b P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:57.368676 25907 tablet_bootstrap.cc:492] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.369017 25907 tablet_bootstrap.cc:492] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap complete.
I20240502 07:58:57.369249 25907 ts_tablet_manager.cc:1367] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.030s user 0.028s sys 0.000s
I20240502 07:58:57.369817 25907 raft_consensus.cc:348] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:57.369949 25907 raft_consensus.cc:729] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.370085 25907 consensus_queue.cc:260] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:57.370405 25907 ts_tablet_manager.cc:1398] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240502 07:58:57.370353 22634 consensus_queue.cc:237] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.370925 22654 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-257f0f4dca624c859d3b06cfd4246536 received from {username='slave'} at 127.15.129.133:55755
I20240502 07:58:57.371006 22654 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-257f0f4dca624c859d3b06cfd4246536 on tablet 257f0f4dca624c859d3b06cfd4246536 with peer 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:58:57.372331 22372 raft_consensus.cc:1261] T 41b4d8a50f684c19988a4cf1ee40921c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:57.372733 22503 raft_consensus.cc:1261] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:57.373175 24462 consensus_queue.cc:1035] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:57.373589 25587 consensus_queue.cc:1035] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:57.375732 24462 raft_consensus.cc:2932] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:57.376247 22503 raft_consensus.cc:2932] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
W20240502 07:58:57.377672 22570 consensus_peers.cc:485] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f -> Peer 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): Couldn't send request to peer 1aecef81733c47b3b468504934357231. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 41b4d8a50f684c19988a4cf1ee40921c. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:57.378067 22372 raft_consensus.cc:2932] T 41b4d8a50f684c19988a4cf1ee40921c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:57.378866 22221 catalog_manager.cc:5484] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:57.394708 25877 ts_tablet_manager.cc:902] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055: Initiating tablet copy from peer fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:58:57.395181 25877 tablet_copy_client.cc:308] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.130:39625
I20240502 07:58:57.400060 22523 tablet_copy_service.cc:140] P fabddb16c85944648a710ec52fc7244c: Received BeginTabletCopySession request for tablet d4c1fadf79ab475b83e3303918f96d46 from peer 227eec4b5f014d349661f6e0b39b9055 ({username='slave'} at 127.15.129.135:36867)
I20240502 07:58:57.400197 22523 tablet_copy_service.cc:161] P fabddb16c85944648a710ec52fc7244c: Beginning new tablet copy session on tablet d4c1fadf79ab475b83e3303918f96d46 from peer 227eec4b5f014d349661f6e0b39b9055 at {username='slave'} at 127.15.129.135:36867: session id = 227eec4b5f014d349661f6e0b39b9055-d4c1fadf79ab475b83e3303918f96d46
I20240502 07:58:57.401386 22523 tablet_copy_source_session.cc:205] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.402988 25877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4c1fadf79ab475b83e3303918f96d46. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.403007 22765 consensus_queue.cc:237] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.404408 22372 raft_consensus.cc:1261] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:57.404842 22634 raft_consensus.cc:1261] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:58:57.404942 25548 consensus_queue.cc:1035] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:57.405262 25877 tablet_copy_client.cc:783] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 0 data blocks...
W20240502 07:58:57.405416 22701 consensus_peers.cc:485] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 -> Peer 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): Couldn't send request to peer 43930dee0cd5498fa8e357a34dd71432. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6d956ab18c8e479a9211afa3563dbe6f. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:58:57.405453 25877 tablet_copy_client.cc:647] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.405640 25548 consensus_queue.cc:1035] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:58:57.406945 25877 tablet_copy_client.cc:523] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.408093 25548 raft_consensus.cc:2932] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:57.408300 25877 tablet_bootstrap.cc:492] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:58:57.410251 22372 raft_consensus.cc:2932] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:57.410892 22227 catalog_manager.cc:5484] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:57.413646 22634 raft_consensus.cc:2932] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:57.420099 25823 ts_tablet_manager.cc:902] T 4e202e9cf0154eb9814a7e3ad93f1e42 P 1aecef81733c47b3b468504934357231: Initiating tablet copy from peer f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:57.420529 25823 tablet_copy_client.cc:308] T 4e202e9cf0154eb9814a7e3ad93f1e42 P 1aecef81733c47b3b468504934357231: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.132:41673
I20240502 07:58:57.420949 22785 tablet_copy_service.cc:140] P f00f3a1876444ad2bf9c919068ab8014: Received BeginTabletCopySession request for tablet 4e202e9cf0154eb9814a7e3ad93f1e42 from peer 1aecef81733c47b3b468504934357231 ({username='slave'} at 127.15.129.134:51231)
I20240502 07:58:57.421048 22785 tablet_copy_service.cc:161] P f00f3a1876444ad2bf9c919068ab8014: Beginning new tablet copy session on tablet 4e202e9cf0154eb9814a7e3ad93f1e42 from peer 1aecef81733c47b3b468504934357231 at {username='slave'} at 127.15.129.134:51231: session id = 1aecef81733c47b3b468504934357231-4e202e9cf0154eb9814a7e3ad93f1e42
I20240502 07:58:57.421993 22785 tablet_copy_source_session.cc:205] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.422888 25823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e202e9cf0154eb9814a7e3ad93f1e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.424953 25823 tablet_copy_client.cc:783] T 4e202e9cf0154eb9814a7e3ad93f1e42 P 1aecef81733c47b3b468504934357231: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.425099 25823 tablet_copy_client.cc:647] T 4e202e9cf0154eb9814a7e3ad93f1e42 P 1aecef81733c47b3b468504934357231: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.426505 25823 tablet_copy_client.cc:523] T 4e202e9cf0154eb9814a7e3ad93f1e42 P 1aecef81733c47b3b468504934357231: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.428287 25823 tablet_bootstrap.cc:492] T 4e202e9cf0154eb9814a7e3ad93f1e42 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:58:57.434396 25877 tablet_bootstrap.cc:492] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.434684 25877 tablet_bootstrap.cc:492] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap complete.
I20240502 07:58:57.434863 25877 ts_tablet_manager.cc:1367] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.027s user 0.020s sys 0.004s
I20240502 07:58:57.435286 22226 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:58:57.435299 25877 raft_consensus.cc:348] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:57.435405 25877 raft_consensus.cc:729] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.435621 25877 consensus_queue.cc:260] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:57.435885 25877 ts_tablet_manager.cc:1398] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:58:57.436575 22523 tablet_copy_service.cc:342] P fabddb16c85944648a710ec52fc7244c: Request end of tablet copy session 227eec4b5f014d349661f6e0b39b9055-d4c1fadf79ab475b83e3303918f96d46 received from {username='slave'} at 127.15.129.135:36867
I20240502 07:58:57.436671 22523 tablet_copy_service.cc:433] P fabddb16c85944648a710ec52fc7244c: ending tablet copy session 227eec4b5f014d349661f6e0b39b9055-d4c1fadf79ab475b83e3303918f96d46 on tablet d4c1fadf79ab475b83e3303918f96d46 with peer 227eec4b5f014d349661f6e0b39b9055
I20240502 07:58:57.445925 25921 tablet.cc:1722] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:58:57.446281 25921 tablet.cc:1620] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:58:57.446247 25922 tablet.cc:1722] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:58:57.446353 25513 tablet.cc:1722] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:58:57.446521 25922 tablet.cc:1620] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:58:57.446627 25513 tablet.cc:1620] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
I20240502 07:58:57.448938 25907 ts_tablet_manager.cc:902] T 648a25ebebe44061bca2844ec5b048c5 P 4b4e7eeb7f3f42119c0363a79febc54b: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:57.449455 25907 tablet_copy_client.cc:308] T 648a25ebebe44061bca2844ec5b048c5 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:58:57.449899 22654 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet 648a25ebebe44061bca2844ec5b048c5 from peer 4b4e7eeb7f3f42119c0363a79febc54b ({username='slave'} at 127.15.129.133:55755)
I20240502 07:58:57.449995 22654 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet 648a25ebebe44061bca2844ec5b048c5 from peer 4b4e7eeb7f3f42119c0363a79febc54b at {username='slave'} at 127.15.129.133:55755: session id = 4b4e7eeb7f3f42119c0363a79febc54b-648a25ebebe44061bca2844ec5b048c5
I20240502 07:58:57.450938 22654 tablet_copy_source_session.cc:205] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.451776 25907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 648a25ebebe44061bca2844ec5b048c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.453056 22221 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=49bcb615327b4dbf9ac425599e1cb97b] alter complete (version 2)
I20240502 07:58:57.453833 25907 tablet_copy_client.cc:783] T 648a25ebebe44061bca2844ec5b048c5 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.453972 25907 tablet_copy_client.cc:647] T 648a25ebebe44061bca2844ec5b048c5 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.455300 25907 tablet_copy_client.cc:523] T 648a25ebebe44061bca2844ec5b048c5 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.456738 25907 tablet_bootstrap.cc:492] T 648a25ebebe44061bca2844ec5b048c5 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:58:57.466318 25823 tablet_bootstrap.cc:492] T 4e202e9cf0154eb9814a7e3ad93f1e42 P 1aecef81733c47b3b468504934357231: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.466611 25823 tablet_bootstrap.cc:492] T 4e202e9cf0154eb9814a7e3ad93f1e42 P 1aecef81733c47b3b468504934357231: Bootstrap complete.
I20240502 07:58:57.466776 25823 ts_tablet_manager.cc:1367] T 4e202e9cf0154eb9814a7e3ad93f1e42 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.039s user 0.020s sys 0.004s
I20240502 07:58:57.467190 25823 raft_consensus.cc:348] T 4e202e9cf0154eb9814a7e3ad93f1e42 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.467298 25823 raft_consensus.cc:729] T 4e202e9cf0154eb9814a7e3ad93f1e42 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.467460 25823 consensus_queue.cc:260] T 4e202e9cf0154eb9814a7e3ad93f1e42 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.468358 25823 ts_tablet_manager.cc:1398] T 4e202e9cf0154eb9814a7e3ad93f1e42 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:57.469439 22785 tablet_copy_service.cc:342] P f00f3a1876444ad2bf9c919068ab8014: Request end of tablet copy session 1aecef81733c47b3b468504934357231-4e202e9cf0154eb9814a7e3ad93f1e42 received from {username='slave'} at 127.15.129.134:51231
I20240502 07:58:57.469522 22785 tablet_copy_service.cc:433] P f00f3a1876444ad2bf9c919068ab8014: ending tablet copy session 1aecef81733c47b3b468504934357231-4e202e9cf0154eb9814a7e3ad93f1e42 on tablet 4e202e9cf0154eb9814a7e3ad93f1e42 with peer 1aecef81733c47b3b468504934357231
I20240502 07:58:57.483489 25877 ts_tablet_manager.cc:902] T d9c0c75053be41f59786f1cf4fb3062c P 227eec4b5f014d349661f6e0b39b9055: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:57.483979 25877 tablet_copy_client.cc:308] T d9c0c75053be41f59786f1cf4fb3062c P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:58:57.488121 25907 tablet_bootstrap.cc:492] T 648a25ebebe44061bca2844ec5b048c5 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.488415 25907 tablet_bootstrap.cc:492] T 648a25ebebe44061bca2844ec5b048c5 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap complete.
I20240502 07:58:57.488502 22654 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet d9c0c75053be41f59786f1cf4fb3062c from peer 227eec4b5f014d349661f6e0b39b9055 ({username='slave'} at 127.15.129.135:39233)
I20240502 07:58:57.488605 22654 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet d9c0c75053be41f59786f1cf4fb3062c from peer 227eec4b5f014d349661f6e0b39b9055 at {username='slave'} at 127.15.129.135:39233: session id = 227eec4b5f014d349661f6e0b39b9055-d9c0c75053be41f59786f1cf4fb3062c
I20240502 07:58:57.488620 25907 ts_tablet_manager.cc:1367] T 648a25ebebe44061bca2844ec5b048c5 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.032s user 0.016s sys 0.005s
I20240502 07:58:57.489022 25907 raft_consensus.cc:348] T 648a25ebebe44061bca2844ec5b048c5 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:57.489176 25907 raft_consensus.cc:729] T 648a25ebebe44061bca2844ec5b048c5 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.489380 25907 consensus_queue.cc:260] T 648a25ebebe44061bca2844ec5b048c5 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:57.489689 25907 ts_tablet_manager.cc:1398] T 648a25ebebe44061bca2844ec5b048c5 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20240502 07:58:57.489902 22654 tablet_copy_source_session.cc:205] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.490315 22653 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-648a25ebebe44061bca2844ec5b048c5 received from {username='slave'} at 127.15.129.133:55755
I20240502 07:58:57.490387 22653 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-648a25ebebe44061bca2844ec5b048c5 on tablet 648a25ebebe44061bca2844ec5b048c5 with peer 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:58:57.493269 25877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9c0c75053be41f59786f1cf4fb3062c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.498193 25907 ts_tablet_manager.cc:902] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b: Initiating tablet copy from peer f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:57.498275 25877 tablet_copy_client.cc:783] T d9c0c75053be41f59786f1cf4fb3062c P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.498471 25877 tablet_copy_client.cc:647] T d9c0c75053be41f59786f1cf4fb3062c P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.498642 25907 tablet_copy_client.cc:308] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.132:41673
I20240502 07:58:57.499125 22785 tablet_copy_service.cc:140] P f00f3a1876444ad2bf9c919068ab8014: Received BeginTabletCopySession request for tablet d6dfeeb6b16a4f92bc756bf8933c7fb1 from peer 4b4e7eeb7f3f42119c0363a79febc54b ({username='slave'} at 127.15.129.133:51403)
I20240502 07:58:57.499234 22785 tablet_copy_service.cc:161] P f00f3a1876444ad2bf9c919068ab8014: Beginning new tablet copy session on tablet d6dfeeb6b16a4f92bc756bf8933c7fb1 from peer 4b4e7eeb7f3f42119c0363a79febc54b at {username='slave'} at 127.15.129.133:51403: session id = 4b4e7eeb7f3f42119c0363a79febc54b-d6dfeeb6b16a4f92bc756bf8933c7fb1
I20240502 07:58:57.500739 22785 tablet_copy_source_session.cc:205] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.501519 25877 tablet_copy_client.cc:523] T d9c0c75053be41f59786f1cf4fb3062c P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.501624 25907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6dfeeb6b16a4f92bc756bf8933c7fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.502774 25877 tablet_bootstrap.cc:492] T d9c0c75053be41f59786f1cf4fb3062c P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:58:57.503610 25907 tablet_copy_client.cc:783] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.503751 25907 tablet_copy_client.cc:647] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.505956 25907 tablet_copy_client.cc:523] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.513849 25907 tablet_bootstrap.cc:492] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:58:57.535136 25486 consensus_queue.cc:1035] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.536072 25877 tablet_bootstrap.cc:492] T d9c0c75053be41f59786f1cf4fb3062c P 227eec4b5f014d349661f6e0b39b9055: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.536311 25856 ts_tablet_manager.cc:902] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:57.536806 25856 tablet_copy_client.cc:308] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:58:57.537396 25877 tablet_bootstrap.cc:492] T d9c0c75053be41f59786f1cf4fb3062c P 227eec4b5f014d349661f6e0b39b9055: Bootstrap complete.
I20240502 07:58:57.537664 22653 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet c9b0511ea0124d84ac5d3996bd1efcd2 from peer 43930dee0cd5498fa8e357a34dd71432 ({username='slave'} at 127.15.129.136:34023)
I20240502 07:58:57.537786 22653 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet c9b0511ea0124d84ac5d3996bd1efcd2 from peer 43930dee0cd5498fa8e357a34dd71432 at {username='slave'} at 127.15.129.136:34023: session id = 43930dee0cd5498fa8e357a34dd71432-c9b0511ea0124d84ac5d3996bd1efcd2
I20240502 07:58:57.538754 22653 tablet_copy_source_session.cc:205] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.537583 25877 ts_tablet_manager.cc:1367] T d9c0c75053be41f59786f1cf4fb3062c P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.035s user 0.019s sys 0.006s
I20240502 07:58:57.539620 25856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9b0511ea0124d84ac5d3996bd1efcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.541787 25856 tablet_copy_client.cc:783] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.541963 25856 tablet_copy_client.cc:647] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.542488 25877 raft_consensus.cc:348] T d9c0c75053be41f59786f1cf4fb3062c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:57.542618 25877 raft_consensus.cc:729] T d9c0c75053be41f59786f1cf4fb3062c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.542762 25877 consensus_queue.cc:260] T d9c0c75053be41f59786f1cf4fb3062c P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:57.543112 25877 ts_tablet_manager.cc:1398] T d9c0c75053be41f59786f1cf4fb3062c P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240502 07:58:57.544001 25907 tablet_bootstrap.cc:492] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.544207 22654 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session 227eec4b5f014d349661f6e0b39b9055-d9c0c75053be41f59786f1cf4fb3062c received from {username='slave'} at 127.15.129.135:39233
I20240502 07:58:57.544286 22654 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session 227eec4b5f014d349661f6e0b39b9055-d9c0c75053be41f59786f1cf4fb3062c on tablet d9c0c75053be41f59786f1cf4fb3062c with peer 227eec4b5f014d349661f6e0b39b9055
I20240502 07:58:57.544302 25907 tablet_bootstrap.cc:492] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap complete.
I20240502 07:58:57.544478 25907 ts_tablet_manager.cc:1367] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.031s user 0.024s sys 0.001s
I20240502 07:58:57.545219 25907 raft_consensus.cc:348] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:57.545356 25907 raft_consensus.cc:729] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.545485 25907 consensus_queue.cc:260] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:57.545929 25856 tablet_copy_client.cc:523] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.546270 25907 ts_tablet_manager.cc:1398] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:57.547260 25856 tablet_bootstrap.cc:492] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:58:57.547567 22785 tablet_copy_service.cc:342] P f00f3a1876444ad2bf9c919068ab8014: Request end of tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-d6dfeeb6b16a4f92bc756bf8933c7fb1 received from {username='slave'} at 127.15.129.133:51403
I20240502 07:58:57.547641 22785 tablet_copy_service.cc:433] P f00f3a1876444ad2bf9c919068ab8014: ending tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-d6dfeeb6b16a4f92bc756bf8933c7fb1 on tablet d6dfeeb6b16a4f92bc756bf8933c7fb1 with peer 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:58:57.548580 25287 consensus_queue.cc:1035] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.560313 22227 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:58:57.567443 25921 tablet.cc:1722] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:58:57.571300 24932 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:57.571826 25922 tablet.cc:1722] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:58:57.571924 25513 tablet.cc:1722] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:58:57.576047 22227 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=49bcb615327b4dbf9ac425599e1cb97b] alter complete (version 3)
I20240502 07:58:57.583843 25877 ts_tablet_manager.cc:902] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055: Initiating tablet copy from peer f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:57.584321 25877 tablet_copy_client.cc:308] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.132:41673
I20240502 07:58:57.589206 22785 tablet_copy_service.cc:140] P f00f3a1876444ad2bf9c919068ab8014: Received BeginTabletCopySession request for tablet fa14f8f327cb4221aabeedac16d258b6 from peer 227eec4b5f014d349661f6e0b39b9055 ({username='slave'} at 127.15.129.135:39217)
I20240502 07:58:57.589306 22785 tablet_copy_service.cc:161] P f00f3a1876444ad2bf9c919068ab8014: Beginning new tablet copy session on tablet fa14f8f327cb4221aabeedac16d258b6 from peer 227eec4b5f014d349661f6e0b39b9055 at {username='slave'} at 127.15.129.135:39217: session id = 227eec4b5f014d349661f6e0b39b9055-fa14f8f327cb4221aabeedac16d258b6
I20240502 07:58:57.590659 22785 tablet_copy_source_session.cc:205] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.591955 25877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa14f8f327cb4221aabeedac16d258b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.591969 25864 ts_tablet_manager.cc:902] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432: Initiating tablet copy from peer f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:57.592442 25864 tablet_copy_client.cc:308] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.132:41673
I20240502 07:58:57.592876 22785 tablet_copy_service.cc:140] P f00f3a1876444ad2bf9c919068ab8014: Received BeginTabletCopySession request for tablet b9557af8565d4588b119f1915e76c3dc from peer 43930dee0cd5498fa8e357a34dd71432 ({username='slave'} at 127.15.129.136:55029)
I20240502 07:58:57.592957 22785 tablet_copy_service.cc:161] P f00f3a1876444ad2bf9c919068ab8014: Beginning new tablet copy session on tablet b9557af8565d4588b119f1915e76c3dc from peer 43930dee0cd5498fa8e357a34dd71432 at {username='slave'} at 127.15.129.136:55029: session id = 43930dee0cd5498fa8e357a34dd71432-b9557af8565d4588b119f1915e76c3dc
I20240502 07:58:57.593710 22785 tablet_copy_source_session.cc:205] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.594168 25877 tablet_copy_client.cc:783] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.594305 25877 tablet_copy_client.cc:647] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.594439 25864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9557af8565d4588b119f1915e76c3dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.595208 25214 raft_consensus.cc:1203] T ce0f9b48d2db4855a86eb20955fdc853 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:57.595642 25856 tablet_bootstrap.cc:492] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.596355 25856 tablet_bootstrap.cc:492] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap complete.
I20240502 07:58:57.596446 25864 tablet_copy_client.cc:783] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.596527 25856 ts_tablet_manager.cc:1367] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.049s user 0.029s sys 0.000s
I20240502 07:58:57.596613 25864 tablet_copy_client.cc:647] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.597035 25856 raft_consensus.cc:348] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.597157 25856 raft_consensus.cc:729] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.597311 25856 consensus_queue.cc:260] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.599627 25856 ts_tablet_manager.cc:1398] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:57.601545 25877 tablet_copy_client.cc:523] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.601680 22653 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session 43930dee0cd5498fa8e357a34dd71432-c9b0511ea0124d84ac5d3996bd1efcd2 received from {username='slave'} at 127.15.129.136:34023
I20240502 07:58:57.601758 22653 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session 43930dee0cd5498fa8e357a34dd71432-c9b0511ea0124d84ac5d3996bd1efcd2 on tablet c9b0511ea0124d84ac5d3996bd1efcd2 with peer 43930dee0cd5498fa8e357a34dd71432
I20240502 07:58:57.603420 25907 ts_tablet_manager.cc:902] T c2422c666baa48078030c43d1c0938f9 P 4b4e7eeb7f3f42119c0363a79febc54b: Initiating tablet copy from peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:57.603691 25877 tablet_bootstrap.cc:492] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:58:57.603901 25907 tablet_copy_client.cc:308] T c2422c666baa48078030c43d1c0938f9 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.129:38535
I20240502 07:58:57.605659 25864 tablet_copy_client.cc:523] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.607030 25864 tablet_bootstrap.cc:492] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:58:57.607918 24800 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:57.608002 25063 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:57.609234 22392 tablet_copy_service.cc:140] P b533a7d8d97943609c6e28e3e36b78a4: Received BeginTabletCopySession request for tablet c2422c666baa48078030c43d1c0938f9 from peer 4b4e7eeb7f3f42119c0363a79febc54b ({username='slave'} at 127.15.129.133:34117)
I20240502 07:58:57.609360 22392 tablet_copy_service.cc:161] P b533a7d8d97943609c6e28e3e36b78a4: Beginning new tablet copy session on tablet c2422c666baa48078030c43d1c0938f9 from peer 4b4e7eeb7f3f42119c0363a79febc54b at {username='slave'} at 127.15.129.133:34117: session id = 4b4e7eeb7f3f42119c0363a79febc54b-c2422c666baa48078030c43d1c0938f9
I20240502 07:58:57.610316 22392 tablet_copy_source_session.cc:205] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.611932 22612 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:58:57.613543 25907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2422c666baa48078030c43d1c0938f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.616206 22482 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:58:57.616428 25907 tablet_copy_client.cc:783] T c2422c666baa48078030c43d1c0938f9 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.616585 25907 tablet_copy_client.cc:647] T c2422c666baa48078030c43d1c0938f9 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.617731 22352 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:58:57.619294 25194 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:57.619679 25907 tablet_copy_client.cc:523] T c2422c666baa48078030c43d1c0938f9 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.620839 25907 tablet_bootstrap.cc:492] T c2422c666baa48078030c43d1c0938f9 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:58:57.626077 22744 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:58:57.632246 25287 consensus_queue.cc:1035] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.643602 22227 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_6" } modify_external_catalogs: true
I20240502 07:58:57.643750 22227 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_6" } modify_external_catalogs: true
I20240502 07:58:57.649536 22227 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 4 replicas of tablet e6a499fd83554a2c8e3e837aedddb61b
I20240502 07:58:57.649893 22227 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet b420738e4ec14fe6bc9b9adddb86ec46
I20240502 07:58:57.650084 22227 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 4 replicas of tablet b9557af8565d4588b119f1915e76c3dc
I20240502 07:58:57.650727 22744 tablet_service.cc:1514] Processing DeleteTablet for tablet e6a499fd83554a2c8e3e837aedddb61b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:57.650916 22743 tablet_service.cc:1514] Processing DeleteTablet for tablet b9557af8565d4588b119f1915e76c3dc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:57.651021 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet b420738e4ec14fe6bc9b9adddb86ec46 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:57.651118 22350 tablet_service.cc:1514] Processing DeleteTablet for tablet b9557af8565d4588b119f1915e76c3dc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:57.651221 25194 tablet_service.cc:1514] Processing DeleteTablet for tablet b9557af8565d4588b119f1915e76c3dc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:58:57.651557 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet e6a499fd83554a2c8e3e837aedddb61b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:58:57.651687 25812 tablet_replica.cc:330] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:57.651823 25812 raft_consensus.cc:2229] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:57.651885 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet b420738e4ec14fe6bc9b9adddb86ec46 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:58:57.652005 25812 raft_consensus.cc:2258] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.652073 26077 tablet_replica.cc:330] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:58:57.652187 26077 raft_consensus.cc:2229] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:57.652279 26077 raft_consensus.cc:2258] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.652465 26078 tablet_replica.cc:330] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:58:57.652493 25812 ts_tablet_manager.cc:1875] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.652596 26078 raft_consensus.cc:2229] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:57.652712 26078 raft_consensus.cc:2258] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.653147 26078 ts_tablet_manager.cc:1875] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.653826 26077 ts_tablet_manager.cc:1875] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.654599 25812 ts_tablet_manager.cc:1888] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:57.654659 25812 log.cc:1196] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/e6a499fd83554a2c8e3e837aedddb61b
I20240502 07:58:57.654899 25812 ts_tablet_manager.cc:1909] T e6a499fd83554a2c8e3e837aedddb61b P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:57.655081 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet b420738e4ec14fe6bc9b9adddb86ec46 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:57.655241 22352 tablet_service.cc:1514] Processing DeleteTablet for tablet e6a499fd83554a2c8e3e837aedddb61b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:57.655651 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet e6a499fd83554a2c8e3e837aedddb61b (table rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]) successfully deleted
I20240502 07:58:57.655763 22482 tablet_service.cc:1514] Processing DeleteTablet for tablet e6a499fd83554a2c8e3e837aedddb61b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:57.655925 25812 tablet_replica.cc:330] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:57.655956 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet b9557af8565d4588b119f1915e76c3dc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:57.656031 25812 raft_consensus.cc:2229] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:57.656244 25812 raft_consensus.cc:2258] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.656809 25812 ts_tablet_manager.cc:1875] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
W20240502 07:58:57.658771 22217 catalog_manager.cc:4810] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): delete failed for tablet b9557af8565d4588b119f1915e76c3dc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9557af8565d4588b119f1915e76c3dc already in progress: copying tablet
I20240502 07:58:57.660566 25812 ts_tablet_manager.cc:1888] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:57.660738 25812 log.cc:1196] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/b9557af8565d4588b119f1915e76c3dc
I20240502 07:58:57.660943 25486 consensus_queue.cc:1035] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.661473 25812 ts_tablet_manager.cc:1909] T b9557af8565d4588b119f1915e76c3dc P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:57.661918 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet b9557af8565d4588b119f1915e76c3dc (table rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]) successfully deleted
I20240502 07:58:57.662189 25812 tablet_replica.cc:330] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:57.662358 25812 raft_consensus.cc:2229] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:57.662581 25812 raft_consensus.cc:2258] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.664294 25812 ts_tablet_manager.cc:1875] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.666132 25812 ts_tablet_manager.cc:1888] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:57.667213 26077 ts_tablet_manager.cc:1888] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:57.667338 26077 log.cc:1196] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/e6a499fd83554a2c8e3e837aedddb61b
I20240502 07:58:57.667667 26077 ts_tablet_manager.cc:1909] T e6a499fd83554a2c8e3e837aedddb61b P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:58:57.668190 25812 log.cc:1196] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/b420738e4ec14fe6bc9b9adddb86ec46
I20240502 07:58:57.670086 25812 ts_tablet_manager.cc:1909] T b420738e4ec14fe6bc9b9adddb86ec46 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:57.670079 26079 tablet_replica.cc:330] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:57.670228 26079 raft_consensus.cc:2229] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:57.670930 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet b420738e4ec14fe6bc9b9adddb86ec46 (table rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]) successfully deleted
I20240502 07:58:57.671190 26079 raft_consensus.cc:2258] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.671725 26079 ts_tablet_manager.cc:1875] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.672355 26078 ts_tablet_manager.cc:1888] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:57.672489 26078 log.cc:1196] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/b420738e4ec14fe6bc9b9adddb86ec46
I20240502 07:58:57.672828 26078 ts_tablet_manager.cc:1909] T b420738e4ec14fe6bc9b9adddb86ec46 P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:58:57.673476 26075 tablet_replica.cc:330] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:57.673655 26075 raft_consensus.cc:2229] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:57.673844 26075 raft_consensus.cc:2258] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.673923 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet e6a499fd83554a2c8e3e837aedddb61b (table rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]) successfully deleted
I20240502 07:58:57.674224 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet b420738e4ec14fe6bc9b9adddb86ec46 (table rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]) successfully deleted
I20240502 07:58:57.674674 26075 ts_tablet_manager.cc:1875] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.677013 22227 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:58:57.677139 22227 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:58:57.682197 26079 ts_tablet_manager.cc:1888] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:57.682638 26079 log.cc:1196] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/e6a499fd83554a2c8e3e837aedddb61b
I20240502 07:58:57.684019 24952 raft_consensus.cc:1203] T 02449bcab2294956918d1889485cfd5f P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:57.685467 26079 ts_tablet_manager.cc:1909] T e6a499fd83554a2c8e3e837aedddb61b P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:58:57.685600 26075 ts_tablet_manager.cc:1888] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:57.686038 26075 log.cc:1196] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/b9557af8565d4588b119f1915e76c3dc
I20240502 07:58:57.686369 26075 ts_tablet_manager.cc:1909] T b9557af8565d4588b119f1915e76c3dc P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:57.686918 26075 tablet_replica.cc:330] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:57.687078 26075 raft_consensus.cc:2229] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:57.687115 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet b9557af8565d4588b119f1915e76c3dc (table rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]) successfully deleted
I20240502 07:58:57.687214 26075 raft_consensus.cc:2258] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.687331 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet e6a499fd83554a2c8e3e837aedddb61b (table rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]) successfully deleted
I20240502 07:58:57.687769 26075 ts_tablet_manager.cc:1875] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.688706 25877 tablet_bootstrap.cc:492] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.688989 25877 tablet_bootstrap.cc:492] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap complete.
I20240502 07:58:57.689167 25877 ts_tablet_manager.cc:1367] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.086s user 0.027s sys 0.000s
I20240502 07:58:57.689643 25877 raft_consensus.cc:348] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:57.689749 25877 raft_consensus.cc:729] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.691975 26075 ts_tablet_manager.cc:1888] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:57.691931 22227 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 140adf1ca68c4dfc9b3b713ecc7b582c
I20240502 07:58:57.692052 26075 log.cc:1196] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/b420738e4ec14fe6bc9b9adddb86ec46
I20240502 07:58:57.692279 26075 ts_tablet_manager.cc:1909] T b420738e4ec14fe6bc9b9adddb86ec46 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:57.692471 25877 consensus_queue.cc:260] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:57.692771 25194 tablet_service.cc:1514] Processing DeleteTablet for tablet 140adf1ca68c4dfc9b3b713ecc7b582c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:58:57.692960 26076 tablet_replica.cc:330] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:58:57.693249 26076 raft_consensus.cc:2229] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:57.686951 26079 tablet_replica.cc:330] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:57.693392 25877 ts_tablet_manager.cc:1398] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:57.693560 26079 raft_consensus.cc:2229] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:57.693656 26076 raft_consensus.cc:2258] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.693759 26079 raft_consensus.cc:2258] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.694299 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet b420738e4ec14fe6bc9b9adddb86ec46 (table rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]) successfully deleted
I20240502 07:58:57.694521 22785 tablet_copy_service.cc:342] P f00f3a1876444ad2bf9c919068ab8014: Request end of tablet copy session 227eec4b5f014d349661f6e0b39b9055-fa14f8f327cb4221aabeedac16d258b6 received from {username='slave'} at 127.15.129.135:39217
I20240502 07:58:57.694597 22785 tablet_copy_service.cc:433] P f00f3a1876444ad2bf9c919068ab8014: ending tablet copy session 227eec4b5f014d349661f6e0b39b9055-fa14f8f327cb4221aabeedac16d258b6 on tablet fa14f8f327cb4221aabeedac16d258b6 with peer 227eec4b5f014d349661f6e0b39b9055
I20240502 07:58:57.694875 26075 tablet_replica.cc:330] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:57.694989 26075 raft_consensus.cc:2229] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:57.695680 26076 ts_tablet_manager.cc:1875] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.694782 26079 ts_tablet_manager.cc:1875] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.695850 26075 raft_consensus.cc:2258] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.696322 26075 ts_tablet_manager.cc:1875] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.697232 25864 tablet_bootstrap.cc:492] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.697502 25864 tablet_bootstrap.cc:492] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432: Bootstrap complete.
I20240502 07:58:57.697691 25864 ts_tablet_manager.cc:1367] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.091s user 0.023s sys 0.005s
I20240502 07:58:57.698164 25864 raft_consensus.cc:348] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.698215 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet 140adf1ca68c4dfc9b3b713ecc7b582c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:57.698277 26079 ts_tablet_manager.cc:1888] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:57.698279 25864 raft_consensus.cc:729] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.698335 26079 log.cc:1196] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/b9557af8565d4588b119f1915e76c3dc
I20240502 07:58:57.698397 25812 tablet_replica.cc:330] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:57.698438 25864 consensus_queue.cc:260] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.698731 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet 140adf1ca68c4dfc9b3b713ecc7b582c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:58:57.698937 25812 raft_consensus.cc:2229] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:57.699111 25812 raft_consensus.cc:2258] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.699770 26075 ts_tablet_manager.cc:1888] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:57.699856 26076 ts_tablet_manager.cc:1888] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:57.699898 26075 log.cc:1196] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/e6a499fd83554a2c8e3e837aedddb61b
I20240502 07:58:57.699996 26079 ts_tablet_manager.cc:1909] T b9557af8565d4588b119f1915e76c3dc P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:58:57.700030 26076 log.cc:1196] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/140adf1ca68c4dfc9b3b713ecc7b582c
I20240502 07:58:57.700163 26075 ts_tablet_manager.cc:1909] T e6a499fd83554a2c8e3e837aedddb61b P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:57.700376 25194 tablet_service.cc:1514] Processing DeleteTablet for tablet b9557af8565d4588b119f1915e76c3dc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:57 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:58:57.700735 26078 tablet_replica.cc:330] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:58:57.700613 26076 ts_tablet_manager.cc:1909] T 140adf1ca68c4dfc9b3b713ecc7b582c P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:58:57.700753 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet e6a499fd83554a2c8e3e837aedddb61b (table rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]) successfully deleted
I20240502 07:58:57.700801 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet b9557af8565d4588b119f1915e76c3dc (table rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]) successfully deleted
I20240502 07:58:57.701059 26078 raft_consensus.cc:2229] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:57.701125 25864 ts_tablet_manager.cc:1398] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 07:58:57.701197 26078 raft_consensus.cc:2258] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.701591 22785 tablet_copy_service.cc:342] P f00f3a1876444ad2bf9c919068ab8014: Request end of tablet copy session 43930dee0cd5498fa8e357a34dd71432-b9557af8565d4588b119f1915e76c3dc received from {username='slave'} at 127.15.129.136:55029
I20240502 07:58:57.701674 22785 tablet_copy_service.cc:433] P f00f3a1876444ad2bf9c919068ab8014: ending tablet copy session 43930dee0cd5498fa8e357a34dd71432-b9557af8565d4588b119f1915e76c3dc on tablet b9557af8565d4588b119f1915e76c3dc with peer 43930dee0cd5498fa8e357a34dd71432
I20240502 07:58:57.702111 26078 ts_tablet_manager.cc:1875] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.703049 26076 tablet_replica.cc:330] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:58:57.703159 26076 raft_consensus.cc:2229] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:57.703250 26076 raft_consensus.cc:2258] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:57.703368 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 140adf1ca68c4dfc9b3b713ecc7b582c (table rebalancer_dynamic_table_new_name [id=49bcb615327b4dbf9ac425599e1cb97b]) successfully deleted
I20240502 07:58:57.703883 25812 ts_tablet_manager.cc:1875] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.704157 26076 ts_tablet_manager.cc:1875] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:57.705982 25812 ts_tablet_manager.cc:1888] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:57.706058 25812 log.cc:1196] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/140adf1ca68c4dfc9b3b713ecc7b582c
I20240502 07:58:57.706290 25812 ts_tablet_manager.cc:1909] T 140adf1ca68c4dfc9b3b713ecc7b582c P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:57.706300 24952 raft_consensus.cc:1203] T c5a30e1ebd7747eb911fba8437e42077 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:57.707338 26076 ts_tablet_manager.cc:1888] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:57.707437 26076 log.cc:1196] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/b9557af8565d4588b119f1915e76c3dc
I20240502 07:58:57.707746 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 140adf1ca68c4dfc9b3b713ecc7b582c (table rebalancer_dynamic_table_new_name [id=49bcb615327b4dbf9ac425599e1cb97b]) successfully deleted
I20240502 07:58:57.707988 26076 ts_tablet_manager.cc:1909] T b9557af8565d4588b119f1915e76c3dc P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:58:57.708448 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet b9557af8565d4588b119f1915e76c3dc (table rebalance_test_table_6 [id=bef7e70a9994447ca2f4a9a43821470d]) successfully deleted
I20240502 07:58:57.710669 26078 ts_tablet_manager.cc:1888] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:57.710801 26078 log.cc:1196] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/140adf1ca68c4dfc9b3b713ecc7b582c
I20240502 07:58:57.711091 26078 ts_tablet_manager.cc:1909] T 140adf1ca68c4dfc9b3b713ecc7b582c P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:58:57.711654 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 140adf1ca68c4dfc9b3b713ecc7b582c (table rebalancer_dynamic_table_new_name [id=49bcb615327b4dbf9ac425599e1cb97b]) successfully deleted
I20240502 07:58:57.713976 24696 consensus_queue.cc:1035] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.716667 25214 raft_consensus.cc:3037] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.718242 25402 consensus_queue.cc:1035] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.736594 25907 tablet_bootstrap.cc:492] T c2422c666baa48078030c43d1c0938f9 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.736905 25907 tablet_bootstrap.cc:492] T c2422c666baa48078030c43d1c0938f9 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap complete.
I20240502 07:58:57.737186 25907 ts_tablet_manager.cc:1367] T c2422c666baa48078030c43d1c0938f9 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.116s user 0.018s sys 0.004s
I20240502 07:58:57.737637 25907 raft_consensus.cc:348] T c2422c666baa48078030c43d1c0938f9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:57.737756 25907 raft_consensus.cc:729] T c2422c666baa48078030c43d1c0938f9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.737897 25907 consensus_queue.cc:260] T c2422c666baa48078030c43d1c0938f9 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:57.738155 25907 ts_tablet_manager.cc:1398] T c2422c666baa48078030c43d1c0938f9 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.738590 22392 tablet_copy_service.cc:342] P b533a7d8d97943609c6e28e3e36b78a4: Request end of tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-c2422c666baa48078030c43d1c0938f9 received from {username='slave'} at 127.15.129.133:34117
I20240502 07:58:57.738672 22392 tablet_copy_service.cc:433] P b533a7d8d97943609c6e28e3e36b78a4: ending tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-c2422c666baa48078030c43d1c0938f9 on tablet c2422c666baa48078030c43d1c0938f9 with peer 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:58:57.740975 25823 ts_tablet_manager.cc:902] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231: Initiating tablet copy from peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:57.741468 25823 tablet_copy_client.cc:308] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.129:38535
I20240502 07:58:57.742007 22392 tablet_copy_service.cc:140] P b533a7d8d97943609c6e28e3e36b78a4: Received BeginTabletCopySession request for tablet 45925e9713cb4b748efde06f77cf3702 from peer 1aecef81733c47b3b468504934357231 ({username='slave'} at 127.15.129.134:45925)
I20240502 07:58:57.742098 22392 tablet_copy_service.cc:161] P b533a7d8d97943609c6e28e3e36b78a4: Beginning new tablet copy session on tablet 45925e9713cb4b748efde06f77cf3702 from peer 1aecef81733c47b3b468504934357231 at {username='slave'} at 127.15.129.134:45925: session id = 1aecef81733c47b3b468504934357231-45925e9713cb4b748efde06f77cf3702
I20240502 07:58:57.742990 22392 tablet_copy_source_session.cc:205] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.743774 25823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45925e9713cb4b748efde06f77cf3702. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.746138 25823 tablet_copy_client.cc:783] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.746291 25823 tablet_copy_client.cc:647] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.746510 25214 raft_consensus.cc:1203] T 72964b1c935441ca840a46ff79921cc3 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:57.748468 25823 tablet_copy_client.cc:523] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.749748 25823 tablet_bootstrap.cc:492] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:58:57.758729 24820 raft_consensus.cc:1203] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:57.778024 25823 tablet_bootstrap.cc:492] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.778328 25823 tablet_bootstrap.cc:492] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231: Bootstrap complete.
I20240502 07:58:57.778517 25823 ts_tablet_manager.cc:1367] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.029s user 0.024s sys 0.000s
I20240502 07:58:57.778967 25823 raft_consensus.cc:348] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.779093 25823 raft_consensus.cc:729] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.779253 25823 consensus_queue.cc:260] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.780206 25823 ts_tablet_manager.cc:1398] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:57.780707 22392 tablet_copy_service.cc:342] P b533a7d8d97943609c6e28e3e36b78a4: Request end of tablet copy session 1aecef81733c47b3b468504934357231-45925e9713cb4b748efde06f77cf3702 received from {username='slave'} at 127.15.129.134:45925
I20240502 07:58:57.780805 22392 tablet_copy_service.cc:433] P b533a7d8d97943609c6e28e3e36b78a4: ending tablet copy session 1aecef81733c47b3b468504934357231-45925e9713cb4b748efde06f77cf3702 on tablet 45925e9713cb4b748efde06f77cf3702 with peer 1aecef81733c47b3b468504934357231
I20240502 07:58:57.802258 22224 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:58:57.808800 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 36887e0172ad4639a451915783cb9bfd (DEFAULT_TABLE table=rebalancer_dynamic_table [id=35f2ac0a22ae4bcd928769bb3cb93d66]), partition=
I20240502 07:58:57.809089 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36887e0172ad4639a451915783cb9bfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.809177 25194 tablet_service.cc:1467] Processing CreateTablet for tablet 36887e0172ad4639a451915783cb9bfd (DEFAULT_TABLE table=rebalancer_dynamic_table [id=35f2ac0a22ae4bcd928769bb3cb93d66]), partition=
I20240502 07:58:57.809237 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 36887e0172ad4639a451915783cb9bfd (DEFAULT_TABLE table=rebalancer_dynamic_table [id=35f2ac0a22ae4bcd928769bb3cb93d66]), partition=
I20240502 07:58:57.809458 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36887e0172ad4639a451915783cb9bfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.809490 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36887e0172ad4639a451915783cb9bfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.811213 26128 tablet_bootstrap.cc:492] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:58:57.812098 26130 tablet_bootstrap.cc:492] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:58:57.812140 26129 tablet_bootstrap.cc:492] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:57.813084 26129 tablet_bootstrap.cc:654] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.814263 26129 tablet_bootstrap.cc:492] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:57.814348 26129 ts_tablet_manager.cc:1367] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:57.814846 26129 raft_consensus.cc:348] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.814958 26129 raft_consensus.cc:374] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.814992 26129 raft_consensus.cc:729] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.815126 26129 consensus_queue.cc:260] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.815419 26129 ts_tablet_manager.cc:1398] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:57.817101 26128 tablet_bootstrap.cc:654] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.818913 26128 tablet_bootstrap.cc:492] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:58:57.819003 26128 ts_tablet_manager.cc:1367] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:58:57.819334 26128 raft_consensus.cc:348] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.819414 26128 raft_consensus.cc:374] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.819434 26128 raft_consensus.cc:729] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.819551 26128 consensus_queue.cc:260] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.819820 26128 ts_tablet_manager.cc:1398] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.820108 26130 tablet_bootstrap.cc:654] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.821081 26130 tablet_bootstrap.cc:492] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:58:57.821156 26130 ts_tablet_manager.cc:1367] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 07:58:57.821592 26130 raft_consensus.cc:348] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.821702 26130 raft_consensus.cc:374] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.821735 26130 raft_consensus.cc:729] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.821851 26130 consensus_queue.cc:260] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.822122 26130 ts_tablet_manager.cc:1398] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.823031 25823 ts_tablet_manager.cc:902] T 41b4d8a50f684c19988a4cf1ee40921c P 1aecef81733c47b3b468504934357231: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:58:57.823504 25823 tablet_copy_client.cc:308] T 41b4d8a50f684c19988a4cf1ee40921c P 1aecef81733c47b3b468504934357231: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:58:57.829504 22653 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet 41b4d8a50f684c19988a4cf1ee40921c from peer 1aecef81733c47b3b468504934357231 ({username='slave'} at 127.15.129.134:44071)
I20240502 07:58:57.829638 22653 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet 41b4d8a50f684c19988a4cf1ee40921c from peer 1aecef81733c47b3b468504934357231 at {username='slave'} at 127.15.129.134:44071: session id = 1aecef81733c47b3b468504934357231-41b4d8a50f684c19988a4cf1ee40921c
I20240502 07:58:57.830678 22653 tablet_copy_source_session.cc:205] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.833415 25864 ts_tablet_manager.cc:902] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432: Initiating tablet copy from peer f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:57.833973 25864 tablet_copy_client.cc:308] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.132:41673
I20240502 07:58:57.834475 22785 tablet_copy_service.cc:140] P f00f3a1876444ad2bf9c919068ab8014: Received BeginTabletCopySession request for tablet 6d956ab18c8e479a9211afa3563dbe6f from peer 43930dee0cd5498fa8e357a34dd71432 ({username='slave'} at 127.15.129.136:55029)
I20240502 07:58:57.834600 22785 tablet_copy_service.cc:161] P f00f3a1876444ad2bf9c919068ab8014: Beginning new tablet copy session on tablet 6d956ab18c8e479a9211afa3563dbe6f from peer 43930dee0cd5498fa8e357a34dd71432 at {username='slave'} at 127.15.129.136:55029: session id = 43930dee0cd5498fa8e357a34dd71432-6d956ab18c8e479a9211afa3563dbe6f
I20240502 07:58:57.835132 25823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b4d8a50f684c19988a4cf1ee40921c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.835526 22785 tablet_copy_source_session.cc:205] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.836575 25864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d956ab18c8e479a9211afa3563dbe6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.837205 22225 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:57.837554 24932 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:57.840735 25486 raft_consensus.cc:482] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:57.840824 25486 raft_consensus.cc:504] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.841089 25864 tablet_copy_client.cc:783] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.841253 25864 tablet_copy_client.cc:647] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.842492 25864 tablet_copy_client.cc:523] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.844162 25864 tablet_bootstrap.cc:492] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:58:57.845203 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 6c958599492f48b7a8e4946640073d5c (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=7fe3829782fb4a98912517ae49651ba5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.845504 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c958599492f48b7a8e4946640073d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.846024 25486 leader_election.cc:288] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:58:57.846855 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 01d9a73be1fd472db39860e538b9c97e (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=7fe3829782fb4a98912517ae49651ba5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.847128 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d9a73be1fd472db39860e538b9c97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.847666 22743 tablet_service.cc:1467] Processing CreateTablet for tablet af4d275b38ab49e7ac63e2fb9ffd8c07 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=7fe3829782fb4a98912517ae49651ba5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.847870 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af4d275b38ab49e7ac63e2fb9ffd8c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.848017 25823 tablet_copy_client.cc:783] T 41b4d8a50f684c19988a4cf1ee40921c P 1aecef81733c47b3b468504934357231: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.848162 25823 tablet_copy_client.cc:647] T 41b4d8a50f684c19988a4cf1ee40921c P 1aecef81733c47b3b468504934357231: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.849457 26129 tablet_bootstrap.cc:492] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:57.850481 26129 tablet_bootstrap.cc:654] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.851836 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 6c958599492f48b7a8e4946640073d5c (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=7fe3829782fb4a98912517ae49651ba5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.852105 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c958599492f48b7a8e4946640073d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.852303 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 01d9a73be1fd472db39860e538b9c97e (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=7fe3829782fb4a98912517ae49651ba5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.852555 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d9a73be1fd472db39860e538b9c97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.853564 24801 tablet_service.cc:1467] Processing CreateTablet for tablet 01d9a73be1fd472db39860e538b9c97e (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=7fe3829782fb4a98912517ae49651ba5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.853780 24801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d9a73be1fd472db39860e538b9c97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.856498 24932 tablet_service.cc:1467] Processing CreateTablet for tablet af4d275b38ab49e7ac63e2fb9ffd8c07 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=7fe3829782fb4a98912517ae49651ba5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.856766 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af4d275b38ab49e7ac63e2fb9ffd8c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.858287 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:58:57.858942 25083 raft_consensus.cc:1203] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:57.858999 25195 tablet_service.cc:1467] Processing CreateTablet for tablet af4d275b38ab49e7ac63e2fb9ffd8c07 (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=7fe3829782fb4a98912517ae49651ba5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.858999 25194 tablet_service.cc:1467] Processing CreateTablet for tablet 6c958599492f48b7a8e4946640073d5c (DEFAULT_TABLE table=rebalancer_extra_table_4 [id=7fe3829782fb4a98912517ae49651ba5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:57.859319 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af4d275b38ab49e7ac63e2fb9ffd8c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.860845 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36887e0172ad4639a451915783cb9bfd" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:58:57.860980 25214 raft_consensus.cc:2454] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:58:57.861308 24888 leader_election.cc:302] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:58:57.861582 25486 raft_consensus.cc:2781] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:57.861637 25486 raft_consensus.cc:482] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:57.861673 25486 raft_consensus.cc:3037] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.861996 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c958599492f48b7a8e4946640073d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.862705 25486 raft_consensus.cc:504] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.862932 26128 tablet_bootstrap.cc:492] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:58:57.863096 25486 leader_election.cc:288] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:58:57.863910 26128 tablet_bootstrap.cc:654] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.863957 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36887e0172ad4639a451915783cb9bfd" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:58:57.864066 25214 raft_consensus.cc:3037] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.864357 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36887e0172ad4639a451915783cb9bfd" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:57.864497 22765 raft_consensus.cc:2454] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:58:57.864908 25214 raft_consensus.cc:2454] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:58:57.865494 25823 tablet_copy_client.cc:523] T 41b4d8a50f684c19988a4cf1ee40921c P 1aecef81733c47b3b468504934357231: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.865717 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36887e0172ad4639a451915783cb9bfd" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:57.865841 22765 raft_consensus.cc:3037] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.866150 22352 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:58:57.866869 25823 tablet_bootstrap.cc:492] T 41b4d8a50f684c19988a4cf1ee40921c P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:58:57.866859 22765 raft_consensus.cc:2454] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:58:57.867213 24888 leader_election.cc:302] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:58:57.867622 26130 tablet_bootstrap.cc:492] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:58:57.867676 25063 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:58:57.867974 24800 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:57.868448 26130 tablet_bootstrap.cc:654] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.869073 25486 raft_consensus.cc:2781] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:57.869164 25486 raft_consensus.cc:686] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:58:57.869309 25486 consensus_queue.cc:237] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.870339 26157 tablet_bootstrap.cc:492] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:58:57.871066 26129 tablet_bootstrap.cc:492] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:57.871161 26129 ts_tablet_manager.cc:1367] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.002s
I20240502 07:58:57.871601 26129 raft_consensus.cc:348] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:57.871703 26129 raft_consensus.cc:374] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.871733 26129 raft_consensus.cc:729] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.871723 26130 tablet_bootstrap.cc:492] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:58:57.871824 26130 ts_tablet_manager.cc:1367] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:58:57.871838 26129 consensus_queue.cc:260] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:57.872247 26130 raft_consensus.cc:348] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:57.872359 26130 raft_consensus.cc:374] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.872387 26130 raft_consensus.cc:729] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.872488 26130 consensus_queue.cc:260] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:57.872723 26130 ts_tablet_manager.cc:1398] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.874294 22222 catalog_manager.cc:5484] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:57.874493 26129 ts_tablet_manager.cc:1398] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 07:58:57.874672 26129 tablet_bootstrap.cc:492] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:57.875636 26129 tablet_bootstrap.cc:654] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.876019 25194 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:57.877440 25877 ts_tablet_manager.cc:902] T 92162f8f5fcb4baf96fa8004b7b41b9b P 227eec4b5f014d349661f6e0b39b9055: Initiating tablet copy from peer f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:57.877940 25877 tablet_copy_client.cc:308] T 92162f8f5fcb4baf96fa8004b7b41b9b P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.132:41673
I20240502 07:58:57.878543 22785 tablet_copy_service.cc:140] P f00f3a1876444ad2bf9c919068ab8014: Received BeginTabletCopySession request for tablet 92162f8f5fcb4baf96fa8004b7b41b9b from peer 227eec4b5f014d349661f6e0b39b9055 ({username='slave'} at 127.15.129.135:39217)
I20240502 07:58:57.878628 22785 tablet_copy_service.cc:161] P f00f3a1876444ad2bf9c919068ab8014: Beginning new tablet copy session on tablet 92162f8f5fcb4baf96fa8004b7b41b9b from peer 227eec4b5f014d349661f6e0b39b9055 at {username='slave'} at 127.15.129.135:39217: session id = 227eec4b5f014d349661f6e0b39b9055-92162f8f5fcb4baf96fa8004b7b41b9b
I20240502 07:58:57.879488 26157 tablet_bootstrap.cc:654] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.879504 22785 tablet_copy_source_session.cc:205] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.880491 26129 tablet_bootstrap.cc:492] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:57.880558 26129 ts_tablet_manager.cc:1367] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20240502 07:58:57.880997 26129 raft_consensus.cc:348] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:57.881105 26129 raft_consensus.cc:374] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.881142 26129 raft_consensus.cc:729] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.881268 26129 consensus_queue.cc:260] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:57.881562 26129 ts_tablet_manager.cc:1398] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.882727 25877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92162f8f5fcb4baf96fa8004b7b41b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.884016 22612 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:58:57.884974 25877 tablet_copy_client.cc:783] T 92162f8f5fcb4baf96fa8004b7b41b9b P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.885128 25877 tablet_copy_client.cc:647] T 92162f8f5fcb4baf96fa8004b7b41b9b P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.895962 26128 tablet_bootstrap.cc:492] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:58:57.896063 26128 ts_tablet_manager.cc:1367] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20240502 07:58:57.896529 26128 raft_consensus.cc:348] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:57.896637 26128 raft_consensus.cc:374] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.896667 26128 raft_consensus.cc:729] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.896790 26128 consensus_queue.cc:260] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:57.897094 26128 ts_tablet_manager.cc:1398] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.897655 26128 tablet_bootstrap.cc:492] T 6c958599492f48b7a8e4946640073d5c P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:58:57.898635 26128 tablet_bootstrap.cc:654] T 6c958599492f48b7a8e4946640073d5c P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.902784 26128 tablet_bootstrap.cc:492] T 6c958599492f48b7a8e4946640073d5c P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:58:57.902865 26128 ts_tablet_manager.cc:1367] T 6c958599492f48b7a8e4946640073d5c P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:57.903280 26128 raft_consensus.cc:348] T 6c958599492f48b7a8e4946640073d5c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:57.903378 26128 raft_consensus.cc:374] T 6c958599492f48b7a8e4946640073d5c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.903406 26128 raft_consensus.cc:729] T 6c958599492f48b7a8e4946640073d5c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.903524 26128 consensus_queue.cc:260] T 6c958599492f48b7a8e4946640073d5c P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:57.903751 26128 ts_tablet_manager.cc:1398] T 6c958599492f48b7a8e4946640073d5c P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.904649 25907 ts_tablet_manager.cc:902] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b: Initiating tablet copy from peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:57.909121 25907 tablet_copy_client.cc:308] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.129:38535
I20240502 07:58:57.909633 22392 tablet_copy_service.cc:140] P b533a7d8d97943609c6e28e3e36b78a4: Received BeginTabletCopySession request for tablet 4dc697950c8140878b50550fe0ff458d from peer 4b4e7eeb7f3f42119c0363a79febc54b ({username='slave'} at 127.15.129.133:34117)
I20240502 07:58:57.909757 22392 tablet_copy_service.cc:161] P b533a7d8d97943609c6e28e3e36b78a4: Beginning new tablet copy session on tablet 4dc697950c8140878b50550fe0ff458d from peer 4b4e7eeb7f3f42119c0363a79febc54b at {username='slave'} at 127.15.129.133:34117: session id = 4b4e7eeb7f3f42119c0363a79febc54b-4dc697950c8140878b50550fe0ff458d
I20240502 07:58:57.910699 22392 tablet_copy_source_session.cc:205] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:58:57.912233 25907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dc697950c8140878b50550fe0ff458d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:57.913293 22743 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:58:57.916147 25907 tablet_copy_client.cc:783] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 0 data blocks...
I20240502 07:58:57.916328 25907 tablet_copy_client.cc:647] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 1 WAL segments...
I20240502 07:58:57.921574 25877 tablet_copy_client.cc:523] T 92162f8f5fcb4baf96fa8004b7b41b9b P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.923014 25877 tablet_bootstrap.cc:492] T 92162f8f5fcb4baf96fa8004b7b41b9b P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:58:57.926229 24952 raft_consensus.cc:1203] T 4e202e9cf0154eb9814a7e3ad93f1e42 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:57.931592 25823 tablet_bootstrap.cc:492] T 41b4d8a50f684c19988a4cf1ee40921c P 1aecef81733c47b3b468504934357231: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.931610 25214 raft_consensus.cc:1203] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:57.937218 26155 tablet_bootstrap.cc:492] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:58:57.938282 26155 tablet_bootstrap.cc:654] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.939742 26155 tablet_bootstrap.cc:492] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:58:57.939888 26155 ts_tablet_manager.cc:1367] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:57.940542 25907 tablet_copy_client.cc:523] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:58:57.940655 26155 raft_consensus.cc:348] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:57.940938 26155 raft_consensus.cc:374] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.941015 26155 raft_consensus.cc:729] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.941208 26155 consensus_queue.cc:260] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:57.941699 26155 ts_tablet_manager.cc:1398] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:57.942111 25907 tablet_bootstrap.cc:492] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:58:57.942828 26155 tablet_bootstrap.cc:492] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:58:57.943877 26155 tablet_bootstrap.cc:654] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.945000 26155 tablet_bootstrap.cc:492] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:58:57.945077 26155 ts_tablet_manager.cc:1367] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:57.945629 26155 raft_consensus.cc:348] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:57.945724 26155 raft_consensus.cc:374] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.945755 26155 raft_consensus.cc:729] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.945871 26155 consensus_queue.cc:260] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:57.946166 26155 ts_tablet_manager.cc:1398] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.951412 26157 tablet_bootstrap.cc:492] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:58:57.951527 26157 ts_tablet_manager.cc:1367] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.082s user 0.002s sys 0.000s
I20240502 07:58:57.951987 26157 raft_consensus.cc:348] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:57.952087 26157 raft_consensus.cc:374] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.952119 26157 raft_consensus.cc:729] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.952244 26157 consensus_queue.cc:260] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:57.952543 26157 ts_tablet_manager.cc:1398] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.952701 26157 tablet_bootstrap.cc:492] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:58:57.953509 25823 tablet_bootstrap.cc:492] T 41b4d8a50f684c19988a4cf1ee40921c P 1aecef81733c47b3b468504934357231: Bootstrap complete.
I20240502 07:58:57.953608 26157 tablet_bootstrap.cc:654] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:57.953735 25823 ts_tablet_manager.cc:1367] T 41b4d8a50f684c19988a4cf1ee40921c P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.087s user 0.026s sys 0.001s
I20240502 07:58:57.954219 25823 raft_consensus.cc:348] T 41b4d8a50f684c19988a4cf1ee40921c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.954324 25823 raft_consensus.cc:729] T 41b4d8a50f684c19988a4cf1ee40921c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.954478 25823 consensus_queue.cc:260] T 41b4d8a50f684c19988a4cf1ee40921c P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:57.954783 25823 ts_tablet_manager.cc:1398] T 41b4d8a50f684c19988a4cf1ee40921c P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.955394 22653 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session 1aecef81733c47b3b468504934357231-41b4d8a50f684c19988a4cf1ee40921c received from {username='slave'} at 127.15.129.134:44071
I20240502 07:58:57.955480 22653 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session 1aecef81733c47b3b468504934357231-41b4d8a50f684c19988a4cf1ee40921c on tablet 41b4d8a50f684c19988a4cf1ee40921c with peer 1aecef81733c47b3b468504934357231
I20240502 07:58:57.958546 26157 tablet_bootstrap.cc:492] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:58:57.958632 26157 ts_tablet_manager.cc:1367] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:58:57.959077 26157 raft_consensus.cc:348] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:57.959182 26157 raft_consensus.cc:374] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:57.959218 26157 raft_consensus.cc:729] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.959338 26157 consensus_queue.cc:260] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:57.959601 26157 ts_tablet_manager.cc:1398] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:57.978878 25864 tablet_bootstrap.cc:492] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.979290 25864 tablet_bootstrap.cc:492] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432: Bootstrap complete.
I20240502 07:58:57.979769 25864 ts_tablet_manager.cc:1367] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.136s user 0.026s sys 0.000s
I20240502 07:58:57.980266 25864 raft_consensus.cc:348] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.980962 25864 raft_consensus.cc:729] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.981206 25864 consensus_queue.cc:260] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:57.982386 25864 ts_tablet_manager.cc:1398] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:57.982836 22785 tablet_copy_service.cc:342] P f00f3a1876444ad2bf9c919068ab8014: Request end of tablet copy session 43930dee0cd5498fa8e357a34dd71432-6d956ab18c8e479a9211afa3563dbe6f received from {username='slave'} at 127.15.129.136:55029
I20240502 07:58:57.982909 22785 tablet_copy_service.cc:433] P f00f3a1876444ad2bf9c919068ab8014: ending tablet copy session 43930dee0cd5498fa8e357a34dd71432-6d956ab18c8e479a9211afa3563dbe6f on tablet 6d956ab18c8e479a9211afa3563dbe6f with peer 43930dee0cd5498fa8e357a34dd71432
I20240502 07:58:57.986637 22222 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:58:57.987830 25907 tablet_bootstrap.cc:492] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.988116 25907 tablet_bootstrap.cc:492] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap complete.
I20240502 07:58:57.988287 25907 ts_tablet_manager.cc:1367] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.046s user 0.027s sys 0.001s
I20240502 07:58:57.989311 25907 raft_consensus.cc:348] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:57.989477 25907 raft_consensus.cc:729] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.989619 25907 consensus_queue.cc:260] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:57.989992 25907 ts_tablet_manager.cc:1398] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:57.991734 22392 tablet_copy_service.cc:342] P b533a7d8d97943609c6e28e3e36b78a4: Request end of tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-4dc697950c8140878b50550fe0ff458d received from {username='slave'} at 127.15.129.133:34117
I20240502 07:58:57.991816 22392 tablet_copy_service.cc:433] P b533a7d8d97943609c6e28e3e36b78a4: ending tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-4dc697950c8140878b50550fe0ff458d on tablet 4dc697950c8140878b50550fe0ff458d with peer 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:58:57.992106 22765 raft_consensus.cc:1261] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:57.992105 25214 raft_consensus.cc:1261] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:57.992825 25569 raft_consensus.cc:482] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:57.992908 25569 raft_consensus.cc:504] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:57.993019 25486 consensus_queue.cc:1035] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.993259 25569 leader_election.cc:288] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:57.993252 25486 consensus_queue.cc:1035] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:57.993769 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d9a73be1fd472db39860e538b9c97e" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:57.993888 22765 raft_consensus.cc:2454] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:58:57.993887 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d9a73be1fd472db39860e538b9c97e" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:58:57.994009 25083 raft_consensus.cc:2454] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:58:57.994117 24757 leader_election.cc:302] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:57.994578 25877 tablet_bootstrap.cc:492] T 92162f8f5fcb4baf96fa8004b7b41b9b P 227eec4b5f014d349661f6e0b39b9055: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:58:57.995072 25877 tablet_bootstrap.cc:492] T 92162f8f5fcb4baf96fa8004b7b41b9b P 227eec4b5f014d349661f6e0b39b9055: Bootstrap complete.
I20240502 07:58:57.995254 25877 ts_tablet_manager.cc:1367] T 92162f8f5fcb4baf96fa8004b7b41b9b P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.072s user 0.024s sys 0.003s
I20240502 07:58:57.995718 25877 raft_consensus.cc:348] T 92162f8f5fcb4baf96fa8004b7b41b9b P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:57.995832 25877 raft_consensus.cc:729] T 92162f8f5fcb4baf96fa8004b7b41b9b P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:58:57.995991 22225 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 36887e0172ad4639a451915783cb9bfd (table rebalancer_dynamic_table [id=35f2ac0a22ae4bcd928769bb3cb93d66]). Expected version 1 got 0
I20240502 07:58:57.995977 25877 consensus_queue.cc:260] T 92162f8f5fcb4baf96fa8004b7b41b9b P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:57.996934 25569 raft_consensus.cc:2781] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:57.996992 25569 raft_consensus.cc:482] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:57.997032 25569 raft_consensus.cc:3037] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.998123 25569 raft_consensus.cc:504] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:57.998240 22224 catalog_manager.cc:5576] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673) does not have the latest schema for tablet 36887e0172ad4639a451915783cb9bfd (table rebalancer_dynamic_table [id=35f2ac0a22ae4bcd928769bb3cb93d66]). Expected version 1 got 0
I20240502 07:58:57.998498 25569 leader_election.cc:288] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:57.999095 25877 ts_tablet_manager.cc:1398] T 92162f8f5fcb4baf96fa8004b7b41b9b P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:57.999544 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d9a73be1fd472db39860e538b9c97e" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:58:57.999650 25083 raft_consensus.cc:3037] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:57.999943 22785 tablet_copy_service.cc:342] P f00f3a1876444ad2bf9c919068ab8014: Request end of tablet copy session 227eec4b5f014d349661f6e0b39b9055-92162f8f5fcb4baf96fa8004b7b41b9b received from {username='slave'} at 127.15.129.135:39217
I20240502 07:58:58.000015 22785 tablet_copy_service.cc:433] P f00f3a1876444ad2bf9c919068ab8014: ending tablet copy session 227eec4b5f014d349661f6e0b39b9055-92162f8f5fcb4baf96fa8004b7b41b9b on tablet 92162f8f5fcb4baf96fa8004b7b41b9b with peer 227eec4b5f014d349661f6e0b39b9055
I20240502 07:58:58.000782 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d9a73be1fd472db39860e538b9c97e" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:58.000895 22764 raft_consensus.cc:3037] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.000902 25083 raft_consensus.cc:2454] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:58:58.001911 22764 raft_consensus.cc:2454] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:58:58.002069 24820 raft_consensus.cc:1203] T c2422c666baa48078030c43d1c0938f9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:58.002115 24754 leader_election.cc:302] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:58:58.003317 26192 raft_consensus.cc:2781] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:58.004420 26192 raft_consensus.cc:686] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:58:58.004722 26192 consensus_queue.cc:237] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:58.004860 25922 tablet.cc:1722] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:58:58.005151 25922 tablet.cc:1620] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:58:58.008020 26193 tablet.cc:1722] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:58:58.008486 26193 tablet.cc:1620] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:58:58.009380 22224 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=35f2ac0a22ae4bcd928769bb3cb93d66] alter complete (version 1)
I20240502 07:58:58.010622 22225 catalog_manager.cc:5484] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:58:58.012202 25921 tablet.cc:1722] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:58:58.012486 25921 tablet.cc:1620] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:58:58.032514 24820 raft_consensus.cc:1203] T 648a25ebebe44061bca2844ec5b048c5 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:58.067740 25083 raft_consensus.cc:1203] T d9c0c75053be41f59786f1cf4fb3062c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:58.085709 24820 raft_consensus.cc:1203] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:58.115365 22224 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:58:58.121752 26193 tablet.cc:1722] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:58:58.122081 26193 tablet.cc:1620] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:58:58.122402 25921 tablet.cc:1722] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:58:58.122684 25921 tablet.cc:1620] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:58:58.123910 25922 tablet.cc:1722] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:58:58.124193 25922 tablet.cc:1620] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:58:58.126986 22225 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=35f2ac0a22ae4bcd928769bb3cb93d66] alter complete (version 2)
I20240502 07:58:58.135702 24952 raft_consensus.cc:1203] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:58.152550 22225 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_8" } modify_external_catalogs: true
I20240502 07:58:58.152719 22225 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_8" } modify_external_catalogs: true
I20240502 07:58:58.155143 22225 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 4 replicas of tablet c9b0511ea0124d84ac5d3996bd1efcd2
I20240502 07:58:58.155586 22225 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 95952027a0314a2ea3f603974b96befe
I20240502 07:58:58.155810 22352 tablet_service.cc:1514] Processing DeleteTablet for tablet c9b0511ea0124d84ac5d3996bd1efcd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:58.155942 22225 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 12d9f49c05264a6291a7489d1675b697
I20240502 07:58:58.156015 25194 tablet_service.cc:1514] Processing DeleteTablet for tablet c9b0511ea0124d84ac5d3996bd1efcd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:58:58.156044 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet c9b0511ea0124d84ac5d3996bd1efcd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:58.156205 26075 tablet_replica.cc:330] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:58.156247 26079 tablet_replica.cc:330] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:58.156345 26075 raft_consensus.cc:2229] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.156376 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet 12d9f49c05264a6291a7489d1675b697 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:58.156389 26079 raft_consensus.cc:2229] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.156399 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet c9b0511ea0124d84ac5d3996bd1efcd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:58:58.156505 26075 raft_consensus.cc:2258] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.156551 26079 raft_consensus.cc:2258] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.156967 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet 12d9f49c05264a6291a7489d1675b697 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:58:58.157083 26075 ts_tablet_manager.cc:1875] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.157096 26079 ts_tablet_manager.cc:1875] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.157096 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet 12d9f49c05264a6291a7489d1675b697 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:58.157274 25812 tablet_replica.cc:330] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:58.157438 22743 tablet_service.cc:1514] Processing DeleteTablet for tablet 95952027a0314a2ea3f603974b96befe with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:58.157490 25812 raft_consensus.cc:2229] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:58.157599 26078 tablet_replica.cc:330] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:58:58.157687 25812 raft_consensus.cc:2258] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.157723 26078 raft_consensus.cc:2229] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:58.158005 26078 raft_consensus.cc:2258] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.158164 25812 ts_tablet_manager.cc:1875] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.158571 26078 ts_tablet_manager.cc:1875] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.159356 26076 tablet_replica.cc:330] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:58:58.159452 26079 ts_tablet_manager.cc:1888] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:58.159476 26076 raft_consensus.cc:2229] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.159518 26079 log.cc:1196] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/c9b0511ea0124d84ac5d3996bd1efcd2
I20240502 07:58:58.159580 26076 raft_consensus.cc:2258] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.159791 26079 ts_tablet_manager.cc:1909] T c9b0511ea0124d84ac5d3996bd1efcd2 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:58:58.160238 22352 tablet_service.cc:1514] Processing DeleteTablet for tablet 95952027a0314a2ea3f603974b96befe with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:58.160494 25812 ts_tablet_manager.cc:1888] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:58.160562 25812 log.cc:1196] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/12d9f49c05264a6291a7489d1675b697
I20240502 07:58:58.160636 26076 ts_tablet_manager.cc:1875] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.160812 25812 ts_tablet_manager.cc:1909] T 12d9f49c05264a6291a7489d1675b697 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:58.160857 26078 ts_tablet_manager.cc:1888] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:58.160921 26078 log.cc:1196] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/c9b0511ea0124d84ac5d3996bd1efcd2
I20240502 07:58:58.161160 26078 ts_tablet_manager.cc:1909] T c9b0511ea0124d84ac5d3996bd1efcd2 P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:58:58.161298 25812 tablet_replica.cc:330] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:58.161430 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 12d9f49c05264a6291a7489d1675b697 (table rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]) successfully deleted
I20240502 07:58:58.161458 25812 raft_consensus.cc:2229] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.161573 25812 raft_consensus.cc:2258] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.161973 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet 95952027a0314a2ea3f603974b96befe with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:58.162009 25812 ts_tablet_manager.cc:1875] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.162118 26079 tablet_replica.cc:330] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:58.162231 26079 raft_consensus.cc:2229] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.162343 26078 tablet_replica.cc:330] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:58:58.162426 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet c9b0511ea0124d84ac5d3996bd1efcd2 (table rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]) successfully deleted
I20240502 07:58:58.162508 26078 raft_consensus.cc:2229] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.162636 26079 raft_consensus.cc:2258] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.162655 26078 raft_consensus.cc:2258] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.162812 26076 ts_tablet_manager.cc:1888] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:58.162883 26076 log.cc:1196] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/c9b0511ea0124d84ac5d3996bd1efcd2
I20240502 07:58:58.163100 26078 ts_tablet_manager.cc:1875] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.163161 26076 ts_tablet_manager.cc:1909] T c9b0511ea0124d84ac5d3996bd1efcd2 P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:58:58.163218 26079 ts_tablet_manager.cc:1875] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.163960 25083 raft_consensus.cc:1203] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:58.164281 25991 raft_consensus.cc:482] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:58.164314 25812 ts_tablet_manager.cc:1888] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:58.164389 25812 log.cc:1196] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/95952027a0314a2ea3f603974b96befe
I20240502 07:58:58.164355 25991 raft_consensus.cc:504] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:58.164676 25812 ts_tablet_manager.cc:1909] T 95952027a0314a2ea3f603974b96befe P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:58.164819 25991 leader_election.cc:288] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:58.165370 26078 ts_tablet_manager.cc:1888] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:58.165434 26078 log.cc:1196] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/12d9f49c05264a6291a7489d1675b697
I20240502 07:58:58.165732 26078 ts_tablet_manager.cc:1909] T 12d9f49c05264a6291a7489d1675b697 P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:58:58.165804 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af4d275b38ab49e7ac63e2fb9ffd8c07" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:58.165921 22765 raft_consensus.cc:2454] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:58:58.165956 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet c9b0511ea0124d84ac5d3996bd1efcd2 (table rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]) successfully deleted
I20240502 07:58:58.166229 25151 leader_election.cc:302] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:58.166337 26075 ts_tablet_manager.cc:1888] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:58.166410 26075 log.cc:1196] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/c9b0511ea0124d84ac5d3996bd1efcd2
I20240502 07:58:58.166576 25991 raft_consensus.cc:2781] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:58.166636 25991 raft_consensus.cc:482] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:58.166671 25991 raft_consensus.cc:3037] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.166891 26075 ts_tablet_manager.cc:1909] T c9b0511ea0124d84ac5d3996bd1efcd2 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:58.167559 26075 tablet_replica.cc:330] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:58.167609 25991 raft_consensus.cc:504] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:58.167717 26075 raft_consensus.cc:2229] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.167857 26075 raft_consensus.cc:2258] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.168006 25991 leader_election.cc:288] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:58.168359 26075 ts_tablet_manager.cc:1875] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.168864 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet c9b0511ea0124d84ac5d3996bd1efcd2 (table rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]) successfully deleted
I20240502 07:58:58.169023 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 95952027a0314a2ea3f603974b96befe (table rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]) successfully deleted
I20240502 07:58:58.169124 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 12d9f49c05264a6291a7489d1675b697 (table rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]) successfully deleted
I20240502 07:58:58.169306 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af4d275b38ab49e7ac63e2fb9ffd8c07" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:58.169447 22765 raft_consensus.cc:3037] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.170465 22765 raft_consensus.cc:2454] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:58:58.170835 26075 ts_tablet_manager.cc:1888] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:58.170877 25151 leader_election.cc:302] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:58:58.171180 25991 raft_consensus.cc:2781] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:58.171283 25991 raft_consensus.cc:686] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:58:58.171413 25991 consensus_queue.cc:237] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:58.171677 26075 log.cc:1196] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/12d9f49c05264a6291a7489d1675b697
I20240502 07:58:58.171958 26075 ts_tablet_manager.cc:1909] T 12d9f49c05264a6291a7489d1675b697 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:58.172108 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet c9b0511ea0124d84ac5d3996bd1efcd2 (table rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]) successfully deleted
I20240502 07:58:58.172508 26079 ts_tablet_manager.cc:1888] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:58.172801 26079 log.cc:1196] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/95952027a0314a2ea3f603974b96befe
I20240502 07:58:58.173101 26079 ts_tablet_manager.cc:1909] T 95952027a0314a2ea3f603974b96befe P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:58:58.173101 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 12d9f49c05264a6291a7489d1675b697 (table rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]) successfully deleted
I20240502 07:58:58.173310 26075 tablet_replica.cc:330] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:58.173239 22229 catalog_manager.cc:5484] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:58:58.173472 26075 raft_consensus.cc:2229] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:58.173683 26075 raft_consensus.cc:2258] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.174194 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 95952027a0314a2ea3f603974b96befe (table rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]) successfully deleted
I20240502 07:58:58.175177 26075 ts_tablet_manager.cc:1875] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.176226 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af4d275b38ab49e7ac63e2fb9ffd8c07" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:58:58.176368 24952 raft_consensus.cc:2454] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:58:58.176385 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af4d275b38ab49e7ac63e2fb9ffd8c07" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:58:58.176527 24951 raft_consensus.cc:3037] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.177456 24951 raft_consensus.cc:2454] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:58:58.179471 26075 ts_tablet_manager.cc:1888] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:58.179535 26075 log.cc:1196] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/95952027a0314a2ea3f603974b96befe
I20240502 07:58:58.179728 26075 ts_tablet_manager.cc:1909] T 95952027a0314a2ea3f603974b96befe P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:58.180452 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 95952027a0314a2ea3f603974b96befe (table rebalance_test_table_8 [id=c952b8d018e640108111542771b4f142]) successfully deleted
I20240502 07:58:58.198980 25567 raft_consensus.cc:482] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:58.199108 25567 raft_consensus.cc:504] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:58.200206 25567 leader_election.cc:288] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:58:58.204196 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c958599492f48b7a8e4946640073d5c" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:58:58.204232 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c958599492f48b7a8e4946640073d5c" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:58:58.204355 24820 raft_consensus.cc:2454] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:58:58.204355 25214 raft_consensus.cc:2454] T 6c958599492f48b7a8e4946640073d5c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:58:58.204691 25019 leader_election.cc:302] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:58:58.204960 25567 raft_consensus.cc:2781] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:58.205010 25567 raft_consensus.cc:482] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:58.205055 25567 raft_consensus.cc:3037] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.205940 25567 raft_consensus.cc:504] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:58.206271 25567 leader_election.cc:288] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:58:58.206620 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c958599492f48b7a8e4946640073d5c" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:58:58.206732 25214 raft_consensus.cc:3037] T 6c958599492f48b7a8e4946640073d5c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.206854 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c958599492f48b7a8e4946640073d5c" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:58:58.206961 24820 raft_consensus.cc:3037] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.207674 25214 raft_consensus.cc:2454] T 6c958599492f48b7a8e4946640073d5c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:58:58.207774 24820 raft_consensus.cc:2454] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:58:58.207969 25020 leader_election.cc:302] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:58:58.208140 25567 raft_consensus.cc:2781] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:58.208304 25567 raft_consensus.cc:686] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:58:58.208520 25567 consensus_queue.cc:237] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:58.210502 22229 catalog_manager.cc:5484] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:58.230960 22229 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:58:58.232813 24800 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:58.234045 25063 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:58.238652 26193 tablet.cc:1722] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:58:58.240384 25922 tablet.cc:1722] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:58:58.241987 25921 tablet.cc:1722] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:58:58.243886 22352 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:58:58.251720 24932 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:58.252928 22229 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=35f2ac0a22ae4bcd928769bb3cb93d66] alter complete (version 3)
I20240502 07:58:58.260229 25194 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:58.264994 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:58:58.272374 22743 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:58:58.286546 22612 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:58:58.297120 25083 raft_consensus.cc:1203] T 92162f8f5fcb4baf96fa8004b7b41b9b P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:58.310747 24820 raft_consensus.cc:1203] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:58.350275 24951 raft_consensus.cc:1203] T 41b4d8a50f684c19988a4cf1ee40921c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:58.357393 22230 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:58:58.357554 22230 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:58:58.359670 22230 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 36887e0172ad4639a451915783cb9bfd
I20240502 07:58:58.360324 24932 tablet_service.cc:1514] Processing DeleteTablet for tablet 36887e0172ad4639a451915783cb9bfd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:37828
I20240502 07:58:58.360376 25194 tablet_service.cc:1514] Processing DeleteTablet for tablet 36887e0172ad4639a451915783cb9bfd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:58:58.360565 26076 tablet_replica.cc:330] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:58:58.360390 22743 tablet_service.cc:1514] Processing DeleteTablet for tablet 36887e0172ad4639a451915783cb9bfd with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:58.360708 26076 raft_consensus.cc:2229] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.360832 26076 raft_consensus.cc:2258] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.360848 25812 tablet_replica.cc:330] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:58.360939 26283 tablet_replica.cc:330] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:58:58.360983 25812 raft_consensus.cc:2229] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.361063 26283 raft_consensus.cc:2229] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:58.361111 25812 raft_consensus.cc:2258] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.361398 26283 raft_consensus.cc:2258] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.361665 26076 ts_tablet_manager.cc:1875] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.361893 25812 ts_tablet_manager.cc:1875] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.361958 26283 ts_tablet_manager.cc:1875] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.363932 26076 ts_tablet_manager.cc:1888] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:58.364002 26076 log.cc:1196] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/36887e0172ad4639a451915783cb9bfd
I20240502 07:58:58.364030 25812 ts_tablet_manager.cc:1888] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:58.364097 25812 log.cc:1196] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/36887e0172ad4639a451915783cb9bfd
I20240502 07:58:58.364267 26076 ts_tablet_manager.cc:1909] T 36887e0172ad4639a451915783cb9bfd P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:58:58.364363 25812 ts_tablet_manager.cc:1909] T 36887e0172ad4639a451915783cb9bfd P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:58.365022 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 36887e0172ad4639a451915783cb9bfd (table rebalancer_dynamic_table_new_name [id=35f2ac0a22ae4bcd928769bb3cb93d66]) successfully deleted
I20240502 07:58:58.365389 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 36887e0172ad4639a451915783cb9bfd (table rebalancer_dynamic_table_new_name [id=35f2ac0a22ae4bcd928769bb3cb93d66]) successfully deleted
I20240502 07:58:58.366245 26283 ts_tablet_manager.cc:1888] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:58.366313 26283 log.cc:1196] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/36887e0172ad4639a451915783cb9bfd
I20240502 07:58:58.366612 26283 ts_tablet_manager.cc:1909] T 36887e0172ad4639a451915783cb9bfd P 1aecef81733c47b3b468504934357231: Deleting consensus metadata
I20240502 07:58:58.367211 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet 36887e0172ad4639a451915783cb9bfd (table rebalancer_dynamic_table_new_name [id=35f2ac0a22ae4bcd928769bb3cb93d66]) successfully deleted
I20240502 07:58:58.401232 25063 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20240502 07:58:58.401680 25569 consensus_queue.cc:1035] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:58.405541 24932 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:58.406981 26192 consensus_queue.cc:1035] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:58.411300 22352 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:58:58.413516 24800 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:58.416013 25194 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:58.425992 22743 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:58:58.430734 22612 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:58:58.436605 25214 raft_consensus.cc:1203] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:58:58.441016 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:58:58.461797 22221 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:58:58.467293 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 35e3e5ed591f412fbcd9c019c446e4d1 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=84af9fe4ef6e4b28bae21afe2234ed41]), partition=
I20240502 07:58:58.467413 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 35e3e5ed591f412fbcd9c019c446e4d1 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=84af9fe4ef6e4b28bae21afe2234ed41]), partition=
I20240502 07:58:58.467598 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35e3e5ed591f412fbcd9c019c446e4d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.467689 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35e3e5ed591f412fbcd9c019c446e4d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.469784 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 35e3e5ed591f412fbcd9c019c446e4d1 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=84af9fe4ef6e4b28bae21afe2234ed41]), partition=
I20240502 07:58:58.469837 26300 tablet_bootstrap.cc:492] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:58.470027 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35e3e5ed591f412fbcd9c019c446e4d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.471307 26300 tablet_bootstrap.cc:654] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.474272 26300 tablet_bootstrap.cc:492] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:58.474383 26300 ts_tablet_manager.cc:1367] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:58.474843 26300 raft_consensus.cc:348] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:58.474960 26300 raft_consensus.cc:374] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.474988 26300 raft_consensus.cc:729] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.475117 26300 consensus_queue.cc:260] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:58.475423 26300 ts_tablet_manager.cc:1398] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:58.491403 26302 tablet_bootstrap.cc:492] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:58:58.492703 26302 tablet_bootstrap.cc:654] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.496457 26302 tablet_bootstrap.cc:492] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:58:58.496559 26302 ts_tablet_manager.cc:1367] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:58.497598 26302 raft_consensus.cc:348] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:58.497745 26302 raft_consensus.cc:374] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.497821 26302 raft_consensus.cc:729] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.497978 26302 consensus_queue.cc:260] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:58.498386 26302 ts_tablet_manager.cc:1398] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:58.498648 26304 tablet_bootstrap.cc:492] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:58:58.500617 26304 tablet_bootstrap.cc:654] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.501832 26304 tablet_bootstrap.cc:492] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:58:58.501929 26304 ts_tablet_manager.cc:1367] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:58.502395 26304 raft_consensus.cc:348] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:58.502545 26304 raft_consensus.cc:374] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.502616 26304 raft_consensus.cc:729] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.502806 26304 consensus_queue.cc:260] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:58.503180 26304 ts_tablet_manager.cc:1398] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:58.518570 22372 consensus_queue.cc:237] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:58:58.519881 24951 raft_consensus.cc:1261] T 02449bcab2294956918d1889485cfd5f P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:58:58.519979 22503 raft_consensus.cc:1261] T 02449bcab2294956918d1889485cfd5f P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:58:58.520341 26322 consensus_queue.cc:1035] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:58.520562 25528 consensus_queue.cc:1035] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:58.522161 25528 raft_consensus.cc:2932] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:58.522651 24951 raft_consensus.cc:2932] T 02449bcab2294956918d1889485cfd5f P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:58.525185 22221 catalog_manager.cc:5484] T 02449bcab2294956918d1889485cfd5f P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:58.527240 22503 raft_consensus.cc:2932] T 02449bcab2294956918d1889485cfd5f P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:58:58.529681 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet 02449bcab2294956918d1889485cfd5f with delete_type TABLET_DATA_TOMBSTONED (TS e51b10af49484aaa8f0267aebc1bf27f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47186
I20240502 07:58:58.530447 26078 tablet_replica.cc:330] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:58:58.530647 26078 raft_consensus.cc:2229] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.530799 26078 raft_consensus.cc:2258] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.531286 26078 ts_tablet_manager.cc:1875] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:58.533063 26078 ts_tablet_manager.cc:1888] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:58:58.533111 26078 log.cc:1196] T 02449bcab2294956918d1889485cfd5f P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/02449bcab2294956918d1889485cfd5f
I20240502 07:58:58.533584 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 02449bcab2294956918d1889485cfd5f (table rebalance_test_table_13 [id=490b2fcdef13484180791ae6a0d1907d]) successfully deleted
I20240502 07:58:58.551132 25991 consensus_queue.cc:1035] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:58.554291 25991 consensus_queue.cc:1035] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:58.555279 25567 raft_consensus.cc:482] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:58.555374 25567 raft_consensus.cc:504] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:58.555802 25567 leader_election.cc:288] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:58:58.556272 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35e3e5ed591f412fbcd9c019c446e4d1" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:58:58.556408 24820 raft_consensus.cc:2454] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:58:58.556574 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35e3e5ed591f412fbcd9c019c446e4d1" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:58.556691 22372 raft_consensus.cc:2454] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:58:58.557299 25019 leader_election.cc:302] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:58:58.558100 25567 raft_consensus.cc:2781] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:58.558159 25567 raft_consensus.cc:482] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:58.558203 25567 raft_consensus.cc:3037] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.559198 25567 raft_consensus.cc:504] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:58.559479 25567 leader_election.cc:288] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:58:58.559784 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35e3e5ed591f412fbcd9c019c446e4d1" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:58.559896 22372 raft_consensus.cc:3037] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.560200 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35e3e5ed591f412fbcd9c019c446e4d1" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:58:58.560317 24820 raft_consensus.cc:3037] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.560892 22372 raft_consensus.cc:2454] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:58:58.561506 24820 raft_consensus.cc:2454] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:58:58.561556 25020 leader_election.cc:302] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:58:58.561909 25567 raft_consensus.cc:2781] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:58.561985 25567 raft_consensus.cc:686] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:58:58.562103 25567 consensus_queue.cc:237] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:58.564158 22229 catalog_manager.cc:5484] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:58.590276 26230 consensus_queue.cc:1035] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:58.597484 25567 consensus_queue.cc:1035] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:58.624948 24800 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:58.625381 25063 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:58.626513 25194 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:58.636422 24932 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:58.637360 22352 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:58:58.657970 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:58:58.662875 22222 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_10" } modify_external_catalogs: true
I20240502 07:58:58.663048 22222 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_10" } modify_external_catalogs: true
I20240502 07:58:58.665673 22222 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet d0e0d340b11e4570b298e9ec75bd4e54
I20240502 07:58:58.665947 22222 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 4 replicas of tablet 6d956ab18c8e479a9211afa3563dbe6f
I20240502 07:58:58.666128 22222 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 4 replicas of tablet d4c1fadf79ab475b83e3303918f96d46
I20240502 07:58:58.666951 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d956ab18c8e479a9211afa3563dbe6f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:58:58.667362 26078 tablet_replica.cc:330] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:58:58.667384 22743 tablet_service.cc:1514] Processing DeleteTablet for tablet d0e0d340b11e4570b298e9ec75bd4e54 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:58.667518 26078 raft_consensus.cc:2229] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.667548 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d956ab18c8e479a9211afa3563dbe6f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:58.667548 22744 tablet_service.cc:1514] Processing DeleteTablet for tablet d4c1fadf79ab475b83e3303918f96d46 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:58.667694 25812 tablet_replica.cc:330] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:58.667708 26078 raft_consensus.cc:2258] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.667834 25812 raft_consensus.cc:2229] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.667961 25812 raft_consensus.cc:2258] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.668067 25194 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d956ab18c8e479a9211afa3563dbe6f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:58:58.668175 22744 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:58:58.668248 26078 ts_tablet_manager.cc:1875] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.668393 22352 tablet_service.cc:1514] Processing DeleteTablet for tablet d0e0d340b11e4570b298e9ec75bd4e54 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:58.668484 25812 ts_tablet_manager.cc:1875] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.668576 26076 tablet_replica.cc:330] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:58:58.668700 26076 raft_consensus.cc:2229] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.668807 26076 raft_consensus.cc:2258] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.669273 26076 ts_tablet_manager.cc:1875] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.669489 26075 tablet_replica.cc:330] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:58.669639 26075 raft_consensus.cc:2229] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.669785 26075 raft_consensus.cc:2258] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.670253 26075 ts_tablet_manager.cc:1875] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.670588 26078 ts_tablet_manager.cc:1888] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:58.670684 26078 log.cc:1196] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/6d956ab18c8e479a9211afa3563dbe6f
I20240502 07:58:58.670729 25812 ts_tablet_manager.cc:1888] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:58.670786 25812 log.cc:1196] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/d0e0d340b11e4570b298e9ec75bd4e54
I20240502 07:58:58.670984 26078 ts_tablet_manager.cc:1909] T 6d956ab18c8e479a9211afa3563dbe6f P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:58:58.671036 25812 ts_tablet_manager.cc:1909] T d0e0d340b11e4570b298e9ec75bd4e54 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:58.671609 25812 tablet_replica.cc:330] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:58.671695 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet d4c1fadf79ab475b83e3303918f96d46 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:58:58.671734 25812 raft_consensus.cc:2229] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:58.671941 25812 raft_consensus.cc:2258] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.671940 22350 tablet_service.cc:1514] Processing DeleteTablet for tablet d4c1fadf79ab475b83e3303918f96d46 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:58.672238 22612 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:58:58.672456 25812 ts_tablet_manager.cc:1875] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.672461 26075 ts_tablet_manager.cc:1888] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:58.672520 26075 log.cc:1196] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/d0e0d340b11e4570b298e9ec75bd4e54
I20240502 07:58:58.672781 26075 ts_tablet_manager.cc:1909] T d0e0d340b11e4570b298e9ec75bd4e54 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:58.673393 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d956ab18c8e479a9211afa3563dbe6f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:58.673571 22482 tablet_service.cc:1514] Processing DeleteTablet for tablet d4c1fadf79ab475b83e3303918f96d46 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:58.673739 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet d0e0d340b11e4570b298e9ec75bd4e54 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:58 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:58.673933 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet d0e0d340b11e4570b298e9ec75bd4e54 (table rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]) successfully deleted
I20240502 07:58:58.674088 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 6d956ab18c8e479a9211afa3563dbe6f (table rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]) successfully deleted
I20240502 07:58:58.674400 26079 tablet_replica.cc:330] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:58.674563 26079 raft_consensus.cc:2229] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:58.674798 26079 raft_consensus.cc:2258] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.675307 26079 ts_tablet_manager.cc:1875] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.676730 26076 ts_tablet_manager.cc:1888] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:58.676820 26076 log.cc:1196] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/6d956ab18c8e479a9211afa3563dbe6f
I20240502 07:58:58.677093 26076 ts_tablet_manager.cc:1909] T 6d956ab18c8e479a9211afa3563dbe6f P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:58:58.678694 26079 ts_tablet_manager.cc:1888] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:58.678776 26079 log.cc:1196] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/d4c1fadf79ab475b83e3303918f96d46
I20240502 07:58:58.678889 26383 tablet_replica.cc:330] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:58:58.679028 26383 raft_consensus.cc:2229] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.679050 26079 ts_tablet_manager.cc:1909] T d4c1fadf79ab475b83e3303918f96d46 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:58:58.679332 26383 raft_consensus.cc:2258] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.679582 26079 tablet_replica.cc:330] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:58.679718 26079 raft_consensus.cc:2229] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:58.679781 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet d4c1fadf79ab475b83e3303918f96d46 (table rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]) successfully deleted
I20240502 07:58:58.679920 26079 raft_consensus.cc:2258] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.679998 26383 ts_tablet_manager.cc:1875] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.680114 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet d0e0d340b11e4570b298e9ec75bd4e54 (table rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]) successfully deleted
I20240502 07:58:58.680375 26075 tablet_replica.cc:330] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:58.680405 26079 ts_tablet_manager.cc:1875] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.680573 26075 raft_consensus.cc:2229] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.680810 26075 raft_consensus.cc:2258] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.681676 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 6d956ab18c8e479a9211afa3563dbe6f (table rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]) successfully deleted
I20240502 07:58:58.682658 26383 ts_tablet_manager.cc:1888] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:58.682613 22222 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:58:58.682833 26383 log.cc:1196] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/d4c1fadf79ab475b83e3303918f96d46
I20240502 07:58:58.683121 25812 ts_tablet_manager.cc:1888] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:58.683212 25812 log.cc:1196] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/6d956ab18c8e479a9211afa3563dbe6f
I20240502 07:58:58.683342 26383 ts_tablet_manager.cc:1909] T d4c1fadf79ab475b83e3303918f96d46 P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:58:58.683492 25812 ts_tablet_manager.cc:1909] T 6d956ab18c8e479a9211afa3563dbe6f P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:58.684123 25812 tablet_replica.cc:330] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:58.684527 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet d4c1fadf79ab475b83e3303918f96d46 (table rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]) successfully deleted
I20240502 07:58:58.684976 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 6d956ab18c8e479a9211afa3563dbe6f (table rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]) successfully deleted
I20240502 07:58:58.685516 25812 raft_consensus.cc:2229] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.685655 26079 ts_tablet_manager.cc:1888] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:58.685717 25812 raft_consensus.cc:2258] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.685842 26079 log.cc:1196] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/d0e0d340b11e4570b298e9ec75bd4e54
I20240502 07:58:58.686151 26079 ts_tablet_manager.cc:1909] T d0e0d340b11e4570b298e9ec75bd4e54 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:58:58.686170 25812 ts_tablet_manager.cc:1875] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.686910 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet d0e0d340b11e4570b298e9ec75bd4e54 (table rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]) successfully deleted
I20240502 07:58:58.687090 26075 ts_tablet_manager.cc:1875] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.688439 22372 raft_consensus.cc:1261] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:58.689087 24820 raft_consensus.cc:1261] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:58.689612 25567 consensus_queue.cc:1035] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:58.690346 25567 consensus_queue.cc:1035] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:58.692179 25812 ts_tablet_manager.cc:1888] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:58.692377 25812 log.cc:1196] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/d4c1fadf79ab475b83e3303918f96d46
I20240502 07:58:58.693245 25812 ts_tablet_manager.cc:1909] T d4c1fadf79ab475b83e3303918f96d46 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:58.701660 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet d4c1fadf79ab475b83e3303918f96d46 (table rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]) successfully deleted
I20240502 07:58:58.703615 26075 ts_tablet_manager.cc:1888] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:58.703707 26075 log.cc:1196] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/d4c1fadf79ab475b83e3303918f96d46
I20240502 07:58:58.703985 26075 ts_tablet_manager.cc:1909] T d4c1fadf79ab475b83e3303918f96d46 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:58.704197 22227 catalog_manager.cc:5576] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535) does not have the latest schema for tablet 35e3e5ed591f412fbcd9c019c446e4d1 (table rebalancer_dynamic_table [id=84af9fe4ef6e4b28bae21afe2234ed41]). Expected version 1 got 0
I20240502 07:58:58.704648 26075 tablet_replica.cc:330] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:58.704811 26075 raft_consensus.cc:2229] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.704952 26075 raft_consensus.cc:2258] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.708354 26075 ts_tablet_manager.cc:1875] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:58.711119 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet d4c1fadf79ab475b83e3303918f96d46 (table rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]) successfully deleted
I20240502 07:58:58.712249 22222 catalog_manager.cc:5576] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661) does not have the latest schema for tablet 35e3e5ed591f412fbcd9c019c446e4d1 (table rebalancer_dynamic_table [id=84af9fe4ef6e4b28bae21afe2234ed41]). Expected version 1 got 0
I20240502 07:58:58.714205 26075 ts_tablet_manager.cc:1888] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:58.714341 26075 log.cc:1196] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/6d956ab18c8e479a9211afa3563dbe6f
I20240502 07:58:58.714748 26075 ts_tablet_manager.cc:1909] T 6d956ab18c8e479a9211afa3563dbe6f P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:58.715502 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 6d956ab18c8e479a9211afa3563dbe6f (table rebalance_test_table_10 [id=d581572e1d9e4b579052fa1e17f5a037]) successfully deleted
I20240502 07:58:58.715575 26387 tablet.cc:1722] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:58:58.716020 26387 tablet.cc:1620] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:58:58.721392 22227 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=84af9fe4ef6e4b28bae21afe2234ed41] alter complete (version 1)
I20240502 07:58:58.722080 26388 tablet.cc:1722] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:58:58.722991 22222 catalog_manager.cc:5576] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535) does not have the latest schema for tablet 35e3e5ed591f412fbcd9c019c446e4d1 (table rebalancer_dynamic_table [id=84af9fe4ef6e4b28bae21afe2234ed41]). Expected version 1 got 0
I20240502 07:58:58.723183 26388 tablet.cc:1620] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:58:58.726823 22222 catalog_manager.cc:5576] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661) does not have the latest schema for tablet 35e3e5ed591f412fbcd9c019c446e4d1 (table rebalancer_dynamic_table [id=84af9fe4ef6e4b28bae21afe2234ed41]). Expected version 1 got 0
I20240502 07:58:58.729287 26389 tablet.cc:1722] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:58:58.729732 26389 tablet.cc:1620] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:58:58.749176 22634 consensus_queue.cc:237] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:58.750484 24820 raft_consensus.cc:1261] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:58:58.750562 22502 raft_consensus.cc:1261] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:58:58.751093 25587 consensus_queue.cc:1035] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:58.751291 26243 consensus_queue.cc:1035] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:58.753484 24462 raft_consensus.cc:2932] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:58.753839 22224 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:58.753909 22502 raft_consensus.cc:2932] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:58.753928 24820 raft_consensus.cc:2932] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:58.756629 22222 catalog_manager.cc:5484] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index 4 to 5, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) evicted. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:58.760078 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet 257f0f4dca624c859d3b06cfd4246536 with delete_type TABLET_DATA_TOMBSTONED (TS b533a7d8d97943609c6e28e3e36b78a4 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:58.760319 26075 tablet_replica.cc:330] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:58.760476 26075 raft_consensus.cc:2229] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:58.760632 26075 raft_consensus.cc:2258] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:58.761173 26075 ts_tablet_manager.cc:1875] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:58.764564 26075 ts_tablet_manager.cc:1888] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:58:58.764639 26075 log.cc:1196] T 257f0f4dca624c859d3b06cfd4246536 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/257f0f4dca624c859d3b06cfd4246536
I20240502 07:58:58.765312 25063 tablet_service.cc:1467] Processing CreateTablet for tablet a9c05f7717854c85a316eb76f9965f23 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=ba79b61566954faab0f2bc52c0c0ba84]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.765658 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9c05f7717854c85a316eb76f9965f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.766952 25194 tablet_service.cc:1467] Processing CreateTablet for tablet a9c05f7717854c85a316eb76f9965f23 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=ba79b61566954faab0f2bc52c0c0ba84]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.767221 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9c05f7717854c85a316eb76f9965f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.767361 22612 tablet_service.cc:1467] Processing CreateTablet for tablet 61690f2a278745aeace757c233e75b25 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=ba79b61566954faab0f2bc52c0c0ba84]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.767483 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 288b33b7a5f84af59bde5fae34cbe924 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=ba79b61566954faab0f2bc52c0c0ba84]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.767628 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61690f2a278745aeace757c233e75b25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.767755 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 288b33b7a5f84af59bde5fae34cbe924. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.768314 24800 tablet_service.cc:1467] Processing CreateTablet for tablet a9c05f7717854c85a316eb76f9965f23 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=ba79b61566954faab0f2bc52c0c0ba84]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.768596 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9c05f7717854c85a316eb76f9965f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.769595 24931 tablet_service.cc:1467] Processing CreateTablet for tablet 61690f2a278745aeace757c233e75b25 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=ba79b61566954faab0f2bc52c0c0ba84]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.769853 24931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61690f2a278745aeace757c233e75b25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.770673 26302 tablet_bootstrap.cc:492] T a9c05f7717854c85a316eb76f9965f23 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:58:58.771219 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 257f0f4dca624c859d3b06cfd4246536 (table rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]) successfully deleted
I20240502 07:58:58.771373 22351 tablet_service.cc:1467] Processing CreateTablet for tablet 288b33b7a5f84af59bde5fae34cbe924 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=ba79b61566954faab0f2bc52c0c0ba84]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.771579 26302 tablet_bootstrap.cc:654] T a9c05f7717854c85a316eb76f9965f23 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.771660 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 288b33b7a5f84af59bde5fae34cbe924. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.772040 26304 tablet_bootstrap.cc:492] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:58:58.772106 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 288b33b7a5f84af59bde5fae34cbe924 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=ba79b61566954faab0f2bc52c0c0ba84]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.772351 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 288b33b7a5f84af59bde5fae34cbe924. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.772859 26304 tablet_bootstrap.cc:654] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.767235 25195 tablet_service.cc:1467] Processing CreateTablet for tablet 61690f2a278745aeace757c233e75b25 (DEFAULT_TABLE table=rebalancer_extra_table_6 [id=ba79b61566954faab0f2bc52c0c0ba84]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:58.773887 26300 tablet_bootstrap.cc:492] T 288b33b7a5f84af59bde5fae34cbe924 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:58.773900 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61690f2a278745aeace757c233e75b25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:58.774660 26414 tablet_bootstrap.cc:492] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:58:58.774677 26300 tablet_bootstrap.cc:654] T 288b33b7a5f84af59bde5fae34cbe924 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.775535 26415 tablet_bootstrap.cc:492] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:58:58.775686 26300 tablet_bootstrap.cc:492] T 288b33b7a5f84af59bde5fae34cbe924 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:58.775784 26300 ts_tablet_manager.cc:1367] T 288b33b7a5f84af59bde5fae34cbe924 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:58.775954 26414 tablet_bootstrap.cc:654] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.776300 26300 raft_consensus.cc:348] T 288b33b7a5f84af59bde5fae34cbe924 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:58.776427 26300 raft_consensus.cc:374] T 288b33b7a5f84af59bde5fae34cbe924 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.776465 26300 raft_consensus.cc:729] T 288b33b7a5f84af59bde5fae34cbe924 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.776558 26304 tablet_bootstrap.cc:492] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:58:58.776587 26300 consensus_queue.cc:260] T 288b33b7a5f84af59bde5fae34cbe924 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:58.776645 26304 ts_tablet_manager.cc:1367] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:58:58.776881 26300 ts_tablet_manager.cc:1398] T 288b33b7a5f84af59bde5fae34cbe924 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:58.776962 26415 tablet_bootstrap.cc:654] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.777117 26304 raft_consensus.cc:348] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:58.777215 26304 raft_consensus.cc:374] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.777244 26304 raft_consensus.cc:729] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.777420 26304 consensus_queue.cc:260] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:58.777751 26304 ts_tablet_manager.cc:1398] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:58.780710 26304 tablet_bootstrap.cc:492] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:58:58.781481 26414 tablet_bootstrap.cc:492] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:58:58.781592 26414 ts_tablet_manager.cc:1367] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:58:58.781915 26304 tablet_bootstrap.cc:654] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.782047 26414 raft_consensus.cc:348] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:58.782150 26414 raft_consensus.cc:374] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.782194 26414 raft_consensus.cc:729] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.782325 26414 consensus_queue.cc:260] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:58.782567 26413 tablet_bootstrap.cc:492] T a9c05f7717854c85a316eb76f9965f23 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:58:58.782683 26414 ts_tablet_manager.cc:1398] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:58.783017 26304 tablet_bootstrap.cc:492] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:58:58.783093 26304 ts_tablet_manager.cc:1367] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:58:58.783556 26304 raft_consensus.cc:348] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:58.783649 26304 raft_consensus.cc:374] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.783658 26302 tablet_bootstrap.cc:492] T a9c05f7717854c85a316eb76f9965f23 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:58:58.783684 26304 raft_consensus.cc:729] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.783759 26302 ts_tablet_manager.cc:1367] T a9c05f7717854c85a316eb76f9965f23 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 07:58:58.783807 26304 consensus_queue.cc:260] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:58.784114 26304 ts_tablet_manager.cc:1398] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:58.784250 26302 raft_consensus.cc:348] T a9c05f7717854c85a316eb76f9965f23 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:58.784353 26302 raft_consensus.cc:374] T a9c05f7717854c85a316eb76f9965f23 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.784382 26302 raft_consensus.cc:729] T a9c05f7717854c85a316eb76f9965f23 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.784509 26302 consensus_queue.cc:260] T a9c05f7717854c85a316eb76f9965f23 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:58.784807 26302 ts_tablet_manager.cc:1398] T a9c05f7717854c85a316eb76f9965f23 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:58.788949 26413 tablet_bootstrap.cc:654] T a9c05f7717854c85a316eb76f9965f23 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.790257 26415 tablet_bootstrap.cc:492] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:58:58.790854 26415 ts_tablet_manager.cc:1367] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 07:58:58.791464 26415 raft_consensus.cc:348] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:58.791566 26415 raft_consensus.cc:374] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.791602 26415 raft_consensus.cc:729] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.791744 26415 consensus_queue.cc:260] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:58.792021 26415 ts_tablet_manager.cc:1398] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:58.792186 26415 tablet_bootstrap.cc:492] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:58:58.793112 26415 tablet_bootstrap.cc:654] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.793810 26413 tablet_bootstrap.cc:492] T a9c05f7717854c85a316eb76f9965f23 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:58:58.793910 26413 ts_tablet_manager.cc:1367] T a9c05f7717854c85a316eb76f9965f23 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 07:58:58.794302 26413 raft_consensus.cc:348] T a9c05f7717854c85a316eb76f9965f23 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:58.794404 26413 raft_consensus.cc:374] T a9c05f7717854c85a316eb76f9965f23 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.794406 26415 tablet_bootstrap.cc:492] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:58:58.794431 26413 raft_consensus.cc:729] T a9c05f7717854c85a316eb76f9965f23 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.794476 26415 ts_tablet_manager.cc:1367] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:58.794574 26413 consensus_queue.cc:260] T a9c05f7717854c85a316eb76f9965f23 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:58.794822 26413 ts_tablet_manager.cc:1398] T a9c05f7717854c85a316eb76f9965f23 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:58.794968 26413 tablet_bootstrap.cc:492] T 61690f2a278745aeace757c233e75b25 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:58:58.794919 26415 raft_consensus.cc:348] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:58.795020 26415 raft_consensus.cc:374] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.795053 26415 raft_consensus.cc:729] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.795534 26415 consensus_queue.cc:260] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:58.795823 26413 tablet_bootstrap.cc:654] T 61690f2a278745aeace757c233e75b25 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:58.796011 26415 ts_tablet_manager.cc:1398] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:58.799762 26413 tablet_bootstrap.cc:492] T 61690f2a278745aeace757c233e75b25 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:58:58.799849 26413 ts_tablet_manager.cc:1367] T 61690f2a278745aeace757c233e75b25 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:58:58.800176 26413 raft_consensus.cc:348] T 61690f2a278745aeace757c233e75b25 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:58.800269 26413 raft_consensus.cc:374] T 61690f2a278745aeace757c233e75b25 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:58.800304 26413 raft_consensus.cc:729] T 61690f2a278745aeace757c233e75b25 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:58:58.800465 26413 consensus_queue.cc:260] T 61690f2a278745aeace757c233e75b25 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:58.800721 26413 ts_tablet_manager.cc:1398] T 61690f2a278745aeace757c233e75b25 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:58.828905 22222 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:58:58.836239 26387 tablet.cc:1722] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:58:58.836592 26387 tablet.cc:1620] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:58:58.839599 26388 tablet.cc:1722] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:58:58.839910 26388 tablet.cc:1620] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:58:58.840286 26389 tablet.cc:1722] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:58:58.840593 26389 tablet.cc:1620] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:58:58.844313 22227 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=84af9fe4ef6e4b28bae21afe2234ed41] alter complete (version 2)
I20240502 07:58:58.917047 24462 raft_consensus.cc:482] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:58.917194 24462 raft_consensus.cc:504] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:58.917757 24462 leader_election.cc:288] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:58:58.918229 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61690f2a278745aeace757c233e75b25" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:58:58.918270 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61690f2a278745aeace757c233e75b25" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:58:58.918362 24951 raft_consensus.cc:2454] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:58.918406 25214 raft_consensus.cc:2454] T 61690f2a278745aeace757c233e75b25 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:58:58.918720 22570 leader_election.cc:302] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:58.918881 24462 raft_consensus.cc:2781] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:58.918938 24462 raft_consensus.cc:482] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:58.918990 24462 raft_consensus.cc:3037] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.919807 24462 raft_consensus.cc:504] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:58.920120 24462 leader_election.cc:288] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:58:58.920485 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61690f2a278745aeace757c233e75b25" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:58:58.920495 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61690f2a278745aeace757c233e75b25" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:58:58.920607 25214 raft_consensus.cc:3037] T 61690f2a278745aeace757c233e75b25 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.920607 24951 raft_consensus.cc:3037] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.921571 24951 raft_consensus.cc:2454] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:58.921571 25214 raft_consensus.cc:2454] T 61690f2a278745aeace757c233e75b25 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:58:58.921844 22570 leader_election.cc:302] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:58:58.922256 24462 raft_consensus.cc:2781] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:58.922350 24462 raft_consensus.cc:686] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:58:58.922498 24462 consensus_queue.cc:237] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:58.924245 22225 catalog_manager.cc:5484] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:58.946560 22225 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:58:58.952139 26387 tablet.cc:1722] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:58:58.952522 26388 tablet.cc:1722] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:58:58.952550 26389 tablet.cc:1722] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:58:58.957206 22225 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=84af9fe4ef6e4b28bae21afe2234ed41] alter complete (version 3)
I20240502 07:58:58.992815 26225 raft_consensus.cc:482] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:58.992946 26225 raft_consensus.cc:504] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:58.993384 26225 leader_election.cc:288] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:58.993748 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "288b33b7a5f84af59bde5fae34cbe924" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:58:58.993817 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "288b33b7a5f84af59bde5fae34cbe924" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:58.993897 25083 raft_consensus.cc:2454] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:58:58.993963 22372 raft_consensus.cc:2454] T 288b33b7a5f84af59bde5fae34cbe924 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:58:58.994212 24885 leader_election.cc:302] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
I20240502 07:58:58.994379 26225 raft_consensus.cc:2781] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:58.994451 26225 raft_consensus.cc:482] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:58.994496 26225 raft_consensus.cc:3037] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.995486 26225 raft_consensus.cc:504] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:58.995843 26225 leader_election.cc:288] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:58.996110 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "288b33b7a5f84af59bde5fae34cbe924" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:58:58.996268 25083 raft_consensus.cc:3037] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.996359 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "288b33b7a5f84af59bde5fae34cbe924" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:58.996484 22372 raft_consensus.cc:3037] T 288b33b7a5f84af59bde5fae34cbe924 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:58.997094 25083 raft_consensus.cc:2454] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:58:58.997308 22372 raft_consensus.cc:2454] T 288b33b7a5f84af59bde5fae34cbe924 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:58:58.997397 24885 leader_election.cc:302] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
I20240502 07:58:58.997558 26225 raft_consensus.cc:2781] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:58.997727 26225 raft_consensus.cc:686] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:58:58.997876 26225 consensus_queue.cc:237] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:58:58.999460 22224 catalog_manager.cc:5484] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:58:59.035320 26230 raft_consensus.cc:482] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:59.035486 26230 raft_consensus.cc:504] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:59.035953 26230 leader_election.cc:288] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:58:59.036428 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9c05f7717854c85a316eb76f9965f23" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:58:59.036499 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9c05f7717854c85a316eb76f9965f23" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:58:59.036563 24820 raft_consensus.cc:2454] T a9c05f7717854c85a316eb76f9965f23 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:58:59.036617 25214 raft_consensus.cc:2454] T a9c05f7717854c85a316eb76f9965f23 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:58:59.036903 25019 leader_election.cc:302] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:58:59.037163 26230 raft_consensus.cc:2781] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:59.037214 26230 raft_consensus.cc:482] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:59.037257 26230 raft_consensus.cc:3037] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.038136 26230 raft_consensus.cc:504] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:59.038457 26230 leader_election.cc:288] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:58:59.038756 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9c05f7717854c85a316eb76f9965f23" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:58:59.038780 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9c05f7717854c85a316eb76f9965f23" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:58:59.038887 24820 raft_consensus.cc:3037] T a9c05f7717854c85a316eb76f9965f23 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.038882 25214 raft_consensus.cc:3037] T a9c05f7717854c85a316eb76f9965f23 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.039789 24820 raft_consensus.cc:2454] T a9c05f7717854c85a316eb76f9965f23 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:58:59.039789 25214 raft_consensus.cc:2454] T a9c05f7717854c85a316eb76f9965f23 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:58:59.040073 25020 leader_election.cc:302] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:58:59.040246 26230 raft_consensus.cc:2781] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:59.040356 26230 raft_consensus.cc:686] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:58:59.040529 26230 consensus_queue.cc:237] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:59.042295 22224 catalog_manager.cc:5484] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:59.060400 25063 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:59.061883 22224 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:58:59.062041 22224 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:58:59.063786 24800 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:59.064267 24932 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:59.067965 22224 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 35e3e5ed591f412fbcd9c019c446e4d1
I20240502 07:58:59.068728 24800 tablet_service.cc:1514] Processing DeleteTablet for tablet 35e3e5ed591f412fbcd9c019c446e4d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:58:59.069001 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet 35e3e5ed591f412fbcd9c019c446e4d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:59.069222 26075 tablet_replica.cc:330] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:59.069401 26075 raft_consensus.cc:2229] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.069556 26075 raft_consensus.cc:2258] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.071331 26075 ts_tablet_manager.cc:1875] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.073365 26075 ts_tablet_manager.cc:1888] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:59.073464 26075 log.cc:1196] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/35e3e5ed591f412fbcd9c019c446e4d1
I20240502 07:58:59.073880 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet 35e3e5ed591f412fbcd9c019c446e4d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:58:59.074069 26075 ts_tablet_manager.cc:1909] T 35e3e5ed591f412fbcd9c019c446e4d1 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:59.075004 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 35e3e5ed591f412fbcd9c019c446e4d1 (table rebalancer_dynamic_table_new_name [id=84af9fe4ef6e4b28bae21afe2234ed41]) successfully deleted
I20240502 07:58:59.077435 26383 tablet_replica.cc:330] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:58:59.077584 26383 raft_consensus.cc:2229] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:59.077913 26383 raft_consensus.cc:2258] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.078562 26477 tablet_replica.cc:330] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:58:59.078753 26477 raft_consensus.cc:2229] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.078892 26477 raft_consensus.cc:2258] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.078976 26383 ts_tablet_manager.cc:1875] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.079403 26477 ts_tablet_manager.cc:1875] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.080698 26383 ts_tablet_manager.cc:1888] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:59.080765 26383 log.cc:1196] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/35e3e5ed591f412fbcd9c019c446e4d1
I20240502 07:58:59.080991 26383 ts_tablet_manager.cc:1909] T 35e3e5ed591f412fbcd9c019c446e4d1 P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:58:59.081452 26477 ts_tablet_manager.cc:1888] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:59.081702 26477 log.cc:1196] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/35e3e5ed591f412fbcd9c019c446e4d1
I20240502 07:58:59.082075 26477 ts_tablet_manager.cc:1909] T 35e3e5ed591f412fbcd9c019c446e4d1 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:58:59.083441 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet 35e3e5ed591f412fbcd9c019c446e4d1 (table rebalancer_dynamic_table_new_name [id=84af9fe4ef6e4b28bae21afe2234ed41]) successfully deleted
I20240502 07:58:59.083662 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet 35e3e5ed591f412fbcd9c019c446e4d1 (table rebalancer_dynamic_table_new_name [id=84af9fe4ef6e4b28bae21afe2234ed41]) successfully deleted
I20240502 07:58:59.091274 22612 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:58:59.097487 25195 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:59.099392 22351 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:58:59.102649 22744 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:58:59.114725 22483 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:58:59.169591 22224 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:58:59.170197 22225 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_12" } modify_external_catalogs: true
I20240502 07:58:59.170305 22225 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_12" } modify_external_catalogs: true
I20240502 07:58:59.173862 22225 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 257f0f4dca624c859d3b06cfd4246536
I20240502 07:58:59.174214 22225 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet f9cf9213d36b42238e9d0c1c4c8a840a
I20240502 07:58:59.174346 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet 257f0f4dca624c859d3b06cfd4246536 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:58:59.174459 22225 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 91f8c025d982409db2a1198a3ddf5837
I20240502 07:58:59.175154 22350 tablet_service.cc:1514] Processing DeleteTablet for tablet 91f8c025d982409db2a1198a3ddf5837 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:59.175350 22744 tablet_service.cc:1514] Processing DeleteTablet for tablet f9cf9213d36b42238e9d0c1c4c8a840a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:59.175406 26494 tablet_replica.cc:330] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:58:59.175441 26075 tablet_replica.cc:330] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:59.175558 26075 raft_consensus.cc:2229] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:59.175551 26494 raft_consensus.cc:2229] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:59.175714 26075 raft_consensus.cc:2258] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.175729 26494 raft_consensus.cc:2258] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.175849 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet f9cf9213d36b42238e9d0c1c4c8a840a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:59.176021 24800 tablet_service.cc:1514] Processing DeleteTablet for tablet 257f0f4dca624c859d3b06cfd4246536 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:58:59.176199 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet 91f8c025d982409db2a1198a3ddf5837 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:59.176220 26477 tablet_replica.cc:330] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:58:59.176214 26075 ts_tablet_manager.cc:1875] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.176333 26477 raft_consensus.cc:2229] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.176380 26494 ts_tablet_manager.cc:1875] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.176452 26477 raft_consensus.cc:2258] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.176510 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet 257f0f4dca624c859d3b06cfd4246536 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:59.176592 22482 tablet_service.cc:1514] Processing DeleteTablet for tablet f9cf9213d36b42238e9d0c1c4c8a840a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:59.176915 26079 tablet_replica.cc:330] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:59.177043 26079 raft_consensus.cc:2229] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.177213 26079 raft_consensus.cc:2258] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.177417 26477 ts_tablet_manager.cc:1875] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.177951 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet 91f8c025d982409db2a1198a3ddf5837 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:58:59.178498 26075 ts_tablet_manager.cc:1888] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:59.178570 26075 log.cc:1196] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/91f8c025d982409db2a1198a3ddf5837
I20240502 07:58:59.178561 26079 ts_tablet_manager.cc:1875] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.178836 26075 ts_tablet_manager.cc:1909] T 91f8c025d982409db2a1198a3ddf5837 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:59.179469 26075 tablet_replica.cc:330] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:59.179525 26477 ts_tablet_manager.cc:1888] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20240502 07:58:59.179580 26477 log.cc:1196] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/257f0f4dca624c859d3b06cfd4246536
I20240502 07:58:59.179587 26075 raft_consensus.cc:2229] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:59.179764 26494 ts_tablet_manager.cc:1888] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20240502 07:58:59.179782 26075 raft_consensus.cc:2258] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.179828 26494 log.cc:1196] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/257f0f4dca624c859d3b06cfd4246536
I20240502 07:58:59.179870 26477 ts_tablet_manager.cc:1909] T 257f0f4dca624c859d3b06cfd4246536 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:58:59.180088 26494 ts_tablet_manager.cc:1909] T 257f0f4dca624c859d3b06cfd4246536 P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:58:59.180270 26075 ts_tablet_manager.cc:1875] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.180538 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet 257f0f4dca624c859d3b06cfd4246536 (table rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]) successfully deleted
I20240502 07:58:59.180753 26494 tablet_replica.cc:330] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:58:59.180754 25812 tablet_replica.cc:330] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:59.180816 26079 ts_tablet_manager.cc:1888] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20240502 07:58:59.180876 26079 log.cc:1196] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/257f0f4dca624c859d3b06cfd4246536
I20240502 07:58:59.180888 26494 raft_consensus.cc:2229] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.180887 25812 raft_consensus.cc:2229] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.181015 25812 raft_consensus.cc:2258] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.181038 26494 raft_consensus.cc:2258] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.181133 26079 ts_tablet_manager.cc:1909] T 257f0f4dca624c859d3b06cfd4246536 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:58:59.181852 26494 ts_tablet_manager.cc:1875] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.182106 25812 ts_tablet_manager.cc:1875] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.184093 25812 ts_tablet_manager.cc:1888] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:59.184144 26494 ts_tablet_manager.cc:1888] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:59.184171 25812 log.cc:1196] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/f9cf9213d36b42238e9d0c1c4c8a840a
I20240502 07:58:59.184203 26494 log.cc:1196] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/91f8c025d982409db2a1198a3ddf5837
I20240502 07:58:59.184435 25812 ts_tablet_manager.cc:1909] T f9cf9213d36b42238e9d0c1c4c8a840a P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:59.184489 26494 ts_tablet_manager.cc:1909] T 91f8c025d982409db2a1198a3ddf5837 P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:58:59.184864 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 257f0f4dca624c859d3b06cfd4246536 (table rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]) successfully deleted
I20240502 07:58:59.185011 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 91f8c025d982409db2a1198a3ddf5837 (table rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]) successfully deleted
I20240502 07:58:59.185642 25195 tablet_service.cc:1467] Processing CreateTablet for tablet 9143376336e94ed9b1c75ddbeaf71034 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=725e9a9b5c984fe5a52ece225d99732b]), partition=
I20240502 07:58:59.185694 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 9143376336e94ed9b1c75ddbeaf71034 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=725e9a9b5c984fe5a52ece225d99732b]), partition=
I20240502 07:58:59.185899 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9143376336e94ed9b1c75ddbeaf71034. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.185931 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9143376336e94ed9b1c75ddbeaf71034. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.187503 26075 ts_tablet_manager.cc:1888] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:59.187561 26075 log.cc:1196] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/f9cf9213d36b42238e9d0c1c4c8a840a
I20240502 07:58:59.187758 26075 ts_tablet_manager.cc:1909] T f9cf9213d36b42238e9d0c1c4c8a840a P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:59.188202 25812 tablet_replica.cc:330] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:59.188279 26413 tablet_bootstrap.cc:492] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:58:59.188361 25812 raft_consensus.cc:2229] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.188508 25812 raft_consensus.cc:2258] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.189015 25812 ts_tablet_manager.cc:1875] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.189167 26413 tablet_bootstrap.cc:654] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.189520 26304 tablet_bootstrap.cc:492] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:58:59.190246 26413 tablet_bootstrap.cc:492] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:58:59.190328 26413 ts_tablet_manager.cc:1367] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:59.190384 26304 tablet_bootstrap.cc:654] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.190860 26413 raft_consensus.cc:348] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.190976 26413 raft_consensus.cc:374] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.191020 26413 raft_consensus.cc:729] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.191155 26413 consensus_queue.cc:260] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.191321 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 9143376336e94ed9b1c75ddbeaf71034 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=725e9a9b5c984fe5a52ece225d99732b]), partition=
I20240502 07:58:59.191560 26413 ts_tablet_manager.cc:1398] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:58:59.191661 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9143376336e94ed9b1c75ddbeaf71034. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.192256 25812 ts_tablet_manager.cc:1888] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:59.192345 25812 log.cc:1196] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/91f8c025d982409db2a1198a3ddf5837
I20240502 07:58:59.192566 25812 ts_tablet_manager.cc:1909] T 91f8c025d982409db2a1198a3ddf5837 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:59.193162 26304 tablet_bootstrap.cc:492] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:58:59.193226 26304 ts_tablet_manager.cc:1367] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:58:59.193753 26304 raft_consensus.cc:348] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.193862 26304 raft_consensus.cc:374] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.193904 26304 raft_consensus.cc:729] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.194034 26304 consensus_queue.cc:260] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.194306 26304 ts_tablet_manager.cc:1398] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:59.194593 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 91f8c025d982409db2a1198a3ddf5837 (table rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]) successfully deleted
I20240502 07:58:59.194799 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet f9cf9213d36b42238e9d0c1c4c8a840a (table rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]) successfully deleted
I20240502 07:58:59.194904 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet f9cf9213d36b42238e9d0c1c4c8a840a (table rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]) successfully deleted
I20240502 07:58:59.195003 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 91f8c025d982409db2a1198a3ddf5837 (table rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]) successfully deleted
I20240502 07:58:59.195864 26302 tablet_bootstrap.cc:492] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:58:59.196345 26079 tablet_replica.cc:330] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:59.196473 26079 raft_consensus.cc:2229] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.196573 26079 raft_consensus.cc:2258] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.196874 26302 tablet_bootstrap.cc:654] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.197039 26079 ts_tablet_manager.cc:1875] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.197638 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 257f0f4dca624c859d3b06cfd4246536 (table rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]) successfully deleted
I20240502 07:58:59.198069 26302 tablet_bootstrap.cc:492] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:58:59.198161 26302 ts_tablet_manager.cc:1367] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:58:59.198431 22634 consensus_queue.cc:237] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:59.198622 26302 raft_consensus.cc:348] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.198732 26302 raft_consensus.cc:374] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.198763 26302 raft_consensus.cc:729] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.198873 26302 consensus_queue.cc:260] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.199152 26302 ts_tablet_manager.cc:1398] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:59.200111 25214 raft_consensus.cc:1261] T 72964b1c935441ca840a46ff79921cc3 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:58:59.200141 26079 ts_tablet_manager.cc:1888] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:59.200207 26079 log.cc:1196] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/f9cf9213d36b42238e9d0c1c4c8a840a
I20240502 07:58:59.200516 26079 ts_tablet_manager.cc:1909] T f9cf9213d36b42238e9d0c1c4c8a840a P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:58:59.201239 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet f9cf9213d36b42238e9d0c1c4c8a840a (table rebalance_test_table_12 [id=144806269704437bbac295cdc492d042]) successfully deleted
I20240502 07:58:59.201364 24462 consensus_queue.cc:1035] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:59.201702 22764 raft_consensus.cc:1261] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:58:59.202947 24462 consensus_queue.cc:1035] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:59.204960 24462 raft_consensus.cc:2932] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:59.205304 22765 raft_consensus.cc:2932] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:59.205403 25214 raft_consensus.cc:2932] T 72964b1c935441ca840a46ff79921cc3 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:59.208000 22225 catalog_manager.cc:5484] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:59.212401 22482 tablet_service.cc:1514] Processing DeleteTablet for tablet 72964b1c935441ca840a46ff79921cc3 with delete_type TABLET_DATA_TOMBSTONED (TS fabddb16c85944648a710ec52fc7244c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:59.212668 26079 tablet_replica.cc:330] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:59.212810 26079 raft_consensus.cc:2229] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.212951 26079 raft_consensus.cc:2258] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.213505 26079 ts_tablet_manager.cc:1875] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:59.215713 26079 ts_tablet_manager.cc:1888] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:58:59.215765 26079 log.cc:1196] T 72964b1c935441ca840a46ff79921cc3 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/72964b1c935441ca840a46ff79921cc3
I20240502 07:58:59.216253 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 72964b1c935441ca840a46ff79921cc3 (table rebalance_test_table_13 [id=490b2fcdef13484180791ae6a0d1907d]) successfully deleted
I20240502 07:58:59.320596 26243 consensus_queue.cc:1035] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:59.323724 25587 consensus_queue.cc:1035] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:59.370383 26230 raft_consensus.cc:482] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:59.370581 26230 raft_consensus.cc:504] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.371124 26230 leader_election.cc:288] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:58:59.371451 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9143376336e94ed9b1c75ddbeaf71034" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:58:59.371543 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9143376336e94ed9b1c75ddbeaf71034" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:58:59.371613 24820 raft_consensus.cc:2454] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:58:59.371667 25214 raft_consensus.cc:2454] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:58:59.371930 25019 leader_election.cc:302] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:58:59.372082 26230 raft_consensus.cc:2781] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:59.372143 26230 raft_consensus.cc:482] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:59.372191 26230 raft_consensus.cc:3037] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.373000 26230 raft_consensus.cc:504] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.373358 26230 leader_election.cc:288] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:58:59.373740 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9143376336e94ed9b1c75ddbeaf71034" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:58:59.373792 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9143376336e94ed9b1c75ddbeaf71034" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:58:59.373865 24820 raft_consensus.cc:3037] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.373888 25214 raft_consensus.cc:3037] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.374809 25214 raft_consensus.cc:2454] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:58:59.374809 24820 raft_consensus.cc:2454] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:58:59.375108 25020 leader_election.cc:302] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:58:59.375289 26230 raft_consensus.cc:2781] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:59.375375 26230 raft_consensus.cc:686] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:58:59.375519 26230 consensus_queue.cc:237] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.377233 22225 catalog_manager.cc:5484] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:59.401351 24932 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:59.401937 25063 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:59.404247 24800 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:59.414552 25195 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:59.417577 26225 consensus_queue.cc:1035] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:59.417892 22612 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:58:59.424122 26463 consensus_queue.cc:1035] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:59.428828 22745 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:58:59.430341 22482 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:58:59.459887 22351 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:58:59.495347 22225 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:58:59.499450 24820 raft_consensus.cc:1261] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:59.499862 25214 raft_consensus.cc:1261] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:58:59.500267 26462 consensus_queue.cc:1035] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:59.500566 26462 consensus_queue.cc:1035] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:59.502939 22226 catalog_manager.cc:5576] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661) does not have the latest schema for tablet 9143376336e94ed9b1c75ddbeaf71034 (table rebalancer_dynamic_table [id=725e9a9b5c984fe5a52ece225d99732b]). Expected version 1 got 0
I20240502 07:58:59.503269 22225 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 9143376336e94ed9b1c75ddbeaf71034 (table rebalancer_dynamic_table [id=725e9a9b5c984fe5a52ece225d99732b]). Expected version 1 got 0
I20240502 07:58:59.504376 26579 tablet.cc:1722] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:58:59.504711 26579 tablet.cc:1620] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:58:59.504748 26580 tablet.cc:1722] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:58:59.505012 26580 tablet.cc:1620] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:58:59.507375 22226 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 9143376336e94ed9b1c75ddbeaf71034 (table rebalancer_dynamic_table [id=725e9a9b5c984fe5a52ece225d99732b]). Expected version 1 got 0
I20240502 07:58:59.506263 26581 tablet.cc:1722] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:58:59.508822 26581 tablet.cc:1620] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:58:59.510205 22225 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=725e9a9b5c984fe5a52ece225d99732b] alter complete (version 1)
I20240502 07:58:59.515576 22634 consensus_queue.cc:237] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:58:59.516959 22372 raft_consensus.cc:1261] T ce0f9b48d2db4855a86eb20955fdc853 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:58:59.517300 26243 consensus_queue.cc:1035] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:59.517707 25214 raft_consensus.cc:1261] T ce0f9b48d2db4855a86eb20955fdc853 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:58:59.518081 26243 consensus_queue.cc:1035] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:59.519049 25587 raft_consensus.cc:2932] T ce0f9b48d2db4855a86eb20955fdc853 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:59.519362 22372 raft_consensus.cc:2932] T ce0f9b48d2db4855a86eb20955fdc853 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:59.520136 25214 raft_consensus.cc:2932] T ce0f9b48d2db4855a86eb20955fdc853 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:59.522092 22225 catalog_manager.cc:5484] T ce0f9b48d2db4855a86eb20955fdc853 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:58:59.525619 22482 tablet_service.cc:1514] Processing DeleteTablet for tablet ce0f9b48d2db4855a86eb20955fdc853 with delete_type TABLET_DATA_TOMBSTONED (TS fabddb16c85944648a710ec52fc7244c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:59.526270 26079 tablet_replica.cc:330] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:59.526434 26079 raft_consensus.cc:2229] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.526577 26079 raft_consensus.cc:2258] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.527047 26079 ts_tablet_manager.cc:1875] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:59.528959 26079 ts_tablet_manager.cc:1888] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:58:59.529011 26079 log.cc:1196] T ce0f9b48d2db4855a86eb20955fdc853 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/ce0f9b48d2db4855a86eb20955fdc853
I20240502 07:58:59.529446 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet ce0f9b48d2db4855a86eb20955fdc853 (table rebalance_test_table_1 [id=be3f9b3a6e5147b0afe27972b50219ce]) successfully deleted
I20240502 07:58:59.572227 22226 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:58:59.580166 22351 tablet_service.cc:1467] Processing CreateTablet for tablet f7828cb273c64934853b2cd38368402f (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=72f8b22955164639b2802532262e4e2f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:59.580513 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7828cb273c64934853b2cd38368402f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.580852 25063 tablet_service.cc:1467] Processing CreateTablet for tablet f7828cb273c64934853b2cd38368402f (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=72f8b22955164639b2802532262e4e2f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:59.581048 24932 tablet_service.cc:1467] Processing CreateTablet for tablet f7828cb273c64934853b2cd38368402f (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=72f8b22955164639b2802532262e4e2f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:59.581123 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7828cb273c64934853b2cd38368402f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.581292 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7828cb273c64934853b2cd38368402f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.583709 26304 tablet_bootstrap.cc:492] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:58:59.583707 24931 tablet_service.cc:1467] Processing CreateTablet for tablet c672cd17f991499b92271126e9cf9cb6 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=72f8b22955164639b2802532262e4e2f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:59.583762 22350 tablet_service.cc:1467] Processing CreateTablet for tablet 9f44c905bfaa4db093a1b39b6c21e03c (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=72f8b22955164639b2802532262e4e2f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:59.583940 24931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c672cd17f991499b92271126e9cf9cb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.583982 22350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f44c905bfaa4db093a1b39b6c21e03c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.584704 26304 tablet_bootstrap.cc:654] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.585805 26304 tablet_bootstrap.cc:492] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:58:59.585883 26304 ts_tablet_manager.cc:1367] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:58:59.585902 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 9f44c905bfaa4db093a1b39b6c21e03c (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=72f8b22955164639b2802532262e4e2f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:59.586139 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f44c905bfaa4db093a1b39b6c21e03c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.586256 25195 tablet_service.cc:1467] Processing CreateTablet for tablet c672cd17f991499b92271126e9cf9cb6 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=72f8b22955164639b2802532262e4e2f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:59.586292 26304 raft_consensus.cc:348] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.586383 26304 raft_consensus.cc:374] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.586416 26304 raft_consensus.cc:729] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.586562 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c672cd17f991499b92271126e9cf9cb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.586534 26304 consensus_queue.cc:260] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.586822 26304 ts_tablet_manager.cc:1398] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:58:59.587152 22745 tablet_service.cc:1467] Processing CreateTablet for tablet c672cd17f991499b92271126e9cf9cb6 (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=72f8b22955164639b2802532262e4e2f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:59.587399 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c672cd17f991499b92271126e9cf9cb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.588675 26413 tablet_bootstrap.cc:492] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:58:59.589529 26413 tablet_bootstrap.cc:654] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.590315 26304 tablet_bootstrap.cc:492] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:58:59.591238 26304 tablet_bootstrap.cc:654] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.591955 26630 tablet_bootstrap.cc:492] T c672cd17f991499b92271126e9cf9cb6 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:58:59.590884 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 9f44c905bfaa4db093a1b39b6c21e03c (DEFAULT_TABLE table=rebalancer_extra_table_8 [id=72f8b22955164639b2802532262e4e2f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:58:59.592599 26304 tablet_bootstrap.cc:492] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:58:59.592705 26304 ts_tablet_manager.cc:1367] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:58:59.592725 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f44c905bfaa4db093a1b39b6c21e03c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.592892 26626 tablet_bootstrap.cc:492] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:58:59.593185 26304 raft_consensus.cc:348] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.593300 26304 raft_consensus.cc:374] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.593353 26304 raft_consensus.cc:729] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.593380 26627 tablet_bootstrap.cc:492] T f7828cb273c64934853b2cd38368402f P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:59.593473 26304 consensus_queue.cc:260] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.593730 26304 ts_tablet_manager.cc:1398] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:59.595530 26630 tablet_bootstrap.cc:654] T c672cd17f991499b92271126e9cf9cb6 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.595626 26413 tablet_bootstrap.cc:492] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:58:59.595717 26413 ts_tablet_manager.cc:1367] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20240502 07:58:59.596168 26413 raft_consensus.cc:348] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:59.596282 26413 raft_consensus.cc:374] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.596315 26413 raft_consensus.cc:729] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.596426 26413 consensus_queue.cc:260] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:59.596702 26413 ts_tablet_manager.cc:1398] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:59.597673 26627 tablet_bootstrap.cc:654] T f7828cb273c64934853b2cd38368402f P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.598734 26626 tablet_bootstrap.cc:654] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.603767 26626 tablet_bootstrap.cc:492] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:58:59.603907 26626 ts_tablet_manager.cc:1367] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 07:58:59.604350 26626 raft_consensus.cc:348] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.604522 26626 raft_consensus.cc:374] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.604589 26626 raft_consensus.cc:729] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.604759 26626 consensus_queue.cc:260] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.605659 26626 ts_tablet_manager.cc:1398] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:59.605875 26626 tablet_bootstrap.cc:492] T c672cd17f991499b92271126e9cf9cb6 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:58:59.606951 26230 consensus_queue.cc:1035] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:59.608189 26626 tablet_bootstrap.cc:654] T c672cd17f991499b92271126e9cf9cb6 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.609215 26627 tablet_bootstrap.cc:492] T f7828cb273c64934853b2cd38368402f P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:59.611258 26627 ts_tablet_manager.cc:1367] T f7828cb273c64934853b2cd38368402f P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 07:58:59.611886 26627 raft_consensus.cc:348] T f7828cb273c64934853b2cd38368402f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.613124 26627 raft_consensus.cc:374] T f7828cb273c64934853b2cd38368402f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.613222 26627 raft_consensus.cc:729] T f7828cb273c64934853b2cd38368402f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.613484 26627 consensus_queue.cc:260] T f7828cb273c64934853b2cd38368402f P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.613934 26627 ts_tablet_manager.cc:1398] T f7828cb273c64934853b2cd38368402f P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 07:58:59.614614 26627 tablet_bootstrap.cc:492] T 9f44c905bfaa4db093a1b39b6c21e03c P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:58:59.615604 26627 tablet_bootstrap.cc:654] T 9f44c905bfaa4db093a1b39b6c21e03c P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.615861 26626 tablet_bootstrap.cc:492] T c672cd17f991499b92271126e9cf9cb6 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:58:59.615950 26626 ts_tablet_manager.cc:1367] T c672cd17f991499b92271126e9cf9cb6 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 07:58:59.616421 26626 raft_consensus.cc:348] T c672cd17f991499b92271126e9cf9cb6 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:59.616540 26626 raft_consensus.cc:374] T c672cd17f991499b92271126e9cf9cb6 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.616570 26626 raft_consensus.cc:729] T c672cd17f991499b92271126e9cf9cb6 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.616675 26626 consensus_queue.cc:260] T c672cd17f991499b92271126e9cf9cb6 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:59.616930 26626 ts_tablet_manager.cc:1398] T c672cd17f991499b92271126e9cf9cb6 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:59.617084 26626 tablet_bootstrap.cc:492] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:58:59.618080 26626 tablet_bootstrap.cc:654] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:58:59.620165 26230 consensus_queue.cc:1035] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:58:59.621152 26627 tablet_bootstrap.cc:492] T 9f44c905bfaa4db093a1b39b6c21e03c P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:58:59.621243 26627 ts_tablet_manager.cc:1367] T 9f44c905bfaa4db093a1b39b6c21e03c P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240502 07:58:59.622380 22225 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:58:59.621711 26627 raft_consensus.cc:348] T 9f44c905bfaa4db093a1b39b6c21e03c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.622702 26627 raft_consensus.cc:374] T 9f44c905bfaa4db093a1b39b6c21e03c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.622776 26627 raft_consensus.cc:729] T 9f44c905bfaa4db093a1b39b6c21e03c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.622965 26627 consensus_queue.cc:260] T 9f44c905bfaa4db093a1b39b6c21e03c P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.623414 26627 ts_tablet_manager.cc:1398] T 9f44c905bfaa4db093a1b39b6c21e03c P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:58:59.623492 26626 tablet_bootstrap.cc:492] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:58:59.623592 26626 ts_tablet_manager.cc:1367] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:58:59.624048 26626 raft_consensus.cc:348] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.624154 26626 raft_consensus.cc:374] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.624186 26626 raft_consensus.cc:729] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.624308 26626 consensus_queue.cc:260] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.626395 26626 ts_tablet_manager.cc:1398] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:58:59.629829 26579 tablet.cc:1722] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:58:59.630132 26579 tablet.cc:1620] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:58:59.632709 26630 tablet_bootstrap.cc:492] T c672cd17f991499b92271126e9cf9cb6 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:58:59.632822 26630 ts_tablet_manager.cc:1367] T c672cd17f991499b92271126e9cf9cb6 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.000s
I20240502 07:58:59.633270 26630 raft_consensus.cc:348] T c672cd17f991499b92271126e9cf9cb6 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:59.633399 26630 raft_consensus.cc:374] T c672cd17f991499b92271126e9cf9cb6 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:58:59.633431 26630 raft_consensus.cc:729] T c672cd17f991499b92271126e9cf9cb6 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:58:59.633548 26630 consensus_queue.cc:260] T c672cd17f991499b92271126e9cf9cb6 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:59.633811 26630 ts_tablet_manager.cc:1398] T c672cd17f991499b92271126e9cf9cb6 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:58:59.636174 26581 tablet.cc:1722] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:58:59.636474 26581 tablet.cc:1620] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:58:59.642218 26580 tablet.cc:1722] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:58:59.642535 26580 tablet.cc:1620] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:58:59.650794 24800 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:59.651989 22223 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=725e9a9b5c984fe5a52ece225d99732b] alter complete (version 2)
I20240502 07:58:59.657160 24932 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:59.661146 25195 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:59.662842 22350 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:58:59.673508 25063 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:58:59.674723 22482 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:58:59.681670 22223 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_14" } modify_external_catalogs: true
I20240502 07:58:59.681859 22223 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_14" } modify_external_catalogs: true
I20240502 07:58:59.685556 22612 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:58:59.686690 22745 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:58:59.686731 22223 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 7b935ed00d34452fb6c9d528d5f67cc5
I20240502 07:58:59.687028 22223 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 4 replicas of tablet d6dfeeb6b16a4f92bc756bf8933c7fb1
I20240502 07:58:59.687235 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b935ed00d34452fb6c9d528d5f67cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:59.687270 22223 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet ef6c248c0e15401b9eceeb6a428193dc
I20240502 07:58:59.687397 25812 tablet_replica.cc:330] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:59.687515 25812 raft_consensus.cc:2229] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:59.687721 25812 raft_consensus.cc:2258] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.687698 22350 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b935ed00d34452fb6c9d528d5f67cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:59.687881 26075 tablet_replica.cc:330] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:59.688021 26075 raft_consensus.cc:2229] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.688163 26075 raft_consensus.cc:2258] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.688190 22350 tablet_service.cc:1514] Processing DeleteTablet for tablet d6dfeeb6b16a4f92bc756bf8933c7fb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:59.688218 25812 ts_tablet_manager.cc:1875] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.688642 26075 ts_tablet_manager.cc:1875] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.690333 25812 ts_tablet_manager.cc:1888] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:59.690392 25812 log.cc:1196] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/7b935ed00d34452fb6c9d528d5f67cc5
I20240502 07:58:59.690663 25812 ts_tablet_manager.cc:1909] T 7b935ed00d34452fb6c9d528d5f67cc5 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:59.690706 26075 ts_tablet_manager.cc:1888] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:59.690774 26075 log.cc:1196] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/7b935ed00d34452fb6c9d528d5f67cc5
I20240502 07:58:59.691030 26075 ts_tablet_manager.cc:1909] T 7b935ed00d34452fb6c9d528d5f67cc5 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:59.691051 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet ef6c248c0e15401b9eceeb6a428193dc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:58:59.691458 24800 tablet_service.cc:1514] Processing DeleteTablet for tablet d6dfeeb6b16a4f92bc756bf8933c7fb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:58:59.691504 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet d6dfeeb6b16a4f92bc756bf8933c7fb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:58:59.691598 22482 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b935ed00d34452fb6c9d528d5f67cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:59.691741 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet ef6c248c0e15401b9eceeb6a428193dc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:58:59.691862 26079 tablet_replica.cc:330] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:59.691982 26079 raft_consensus.cc:2229] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.692107 26079 raft_consensus.cc:2258] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.692657 26079 ts_tablet_manager.cc:1875] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.692684 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 7b935ed00d34452fb6c9d528d5f67cc5 (table rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]) successfully deleted
I20240502 07:58:59.692839 26075 tablet_replica.cc:330] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:59.692952 26075 raft_consensus.cc:2229] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.693085 26075 raft_consensus.cc:2258] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.693105 22744 tablet_service.cc:1514] Processing DeleteTablet for tablet ef6c248c0e15401b9eceeb6a428193dc with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:59.693590 26075 ts_tablet_manager.cc:1875] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.693610 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet d6dfeeb6b16a4f92bc756bf8933c7fb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:58:59.694977 26079 ts_tablet_manager.cc:1888] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:59.695051 26079 log.cc:1196] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/7b935ed00d34452fb6c9d528d5f67cc5
I20240502 07:58:59.695266 26659 tablet_replica.cc:330] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:58:59.695320 26079 ts_tablet_manager.cc:1909] T 7b935ed00d34452fb6c9d528d5f67cc5 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:58:59.695396 26659 raft_consensus.cc:2229] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.695542 26659 raft_consensus.cc:2258] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.695794 26075 ts_tablet_manager.cc:1888] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:59.695942 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 7b935ed00d34452fb6c9d528d5f67cc5 (table rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]) successfully deleted
I20240502 07:58:59.696048 26659 ts_tablet_manager.cc:1875] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.696086 26075 log.cc:1196] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/d6dfeeb6b16a4f92bc756bf8933c7fb1
I20240502 07:58:59.696593 26075 ts_tablet_manager.cc:1909] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:59.696753 26079 tablet_replica.cc:330] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:58:59.696872 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 7b935ed00d34452fb6c9d528d5f67cc5 (table rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]) successfully deleted
I20240502 07:58:59.696898 26079 raft_consensus.cc:2229] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:59.697067 25812 tablet_replica.cc:330] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:59.697073 26079 raft_consensus.cc:2258] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.697201 25812 raft_consensus.cc:2229] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.697361 25812 raft_consensus.cc:2258] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.697569 26079 ts_tablet_manager.cc:1875] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.697926 26075 tablet_replica.cc:330] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:58:59.698009 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet d6dfeeb6b16a4f92bc756bf8933c7fb1 (table rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]) successfully deleted
I20240502 07:58:59.698144 26075 raft_consensus.cc:2229] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.698294 26075 raft_consensus.cc:2258] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.698769 26075 ts_tablet_manager.cc:1875] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.699275 26079 ts_tablet_manager.cc:1888] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:59.699345 26079 log.cc:1196] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/ef6c248c0e15401b9eceeb6a428193dc
I20240502 07:58:59.699595 26079 ts_tablet_manager.cc:1909] T ef6c248c0e15401b9eceeb6a428193dc P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:58:59.699988 26659 ts_tablet_manager.cc:1888] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:59.700095 26659 log.cc:1196] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/d6dfeeb6b16a4f92bc756bf8933c7fb1
I20240502 07:58:59.700361 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet ef6c248c0e15401b9eceeb6a428193dc (table rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]) successfully deleted
I20240502 07:58:59.700623 26659 ts_tablet_manager.cc:1909] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:58:59.701447 25812 ts_tablet_manager.cc:1875] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.701962 26075 ts_tablet_manager.cc:1888] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:59.702033 26075 log.cc:1196] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/ef6c248c0e15401b9eceeb6a428193dc
I20240502 07:58:59.702263 26075 ts_tablet_manager.cc:1909] T ef6c248c0e15401b9eceeb6a428193dc P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:58:59.703110 25812 ts_tablet_manager.cc:1888] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:58:59.703187 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet ef6c248c0e15401b9eceeb6a428193dc (table rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]) successfully deleted
I20240502 07:58:59.703572 25812 log.cc:1196] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/ef6c248c0e15401b9eceeb6a428193dc
I20240502 07:58:59.703858 25812 ts_tablet_manager.cc:1909] T ef6c248c0e15401b9eceeb6a428193dc P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:59.704152 25812 tablet_replica.cc:330] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:58:59.704257 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet ef6c248c0e15401b9eceeb6a428193dc (table rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]) successfully deleted
I20240502 07:58:59.704275 25812 raft_consensus.cc:2229] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:59.704461 25812 raft_consensus.cc:2258] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.704944 25812 ts_tablet_manager.cc:1875] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.706032 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet d6dfeeb6b16a4f92bc756bf8933c7fb1 (table rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]) successfully deleted
I20240502 07:58:59.720228 25812 ts_tablet_manager.cc:1888] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:59.720319 25812 log.cc:1196] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/d6dfeeb6b16a4f92bc756bf8933c7fb1
I20240502 07:58:59.720568 25812 ts_tablet_manager.cc:1909] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:58:59.720888 26658 tablet_replica.cc:330] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:58:59.721109 26658 raft_consensus.cc:2229] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.721269 26658 raft_consensus.cc:2258] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.721637 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet d6dfeeb6b16a4f92bc756bf8933c7fb1 (table rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]) successfully deleted
I20240502 07:58:59.721892 26658 ts_tablet_manager.cc:1875] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.723980 26658 ts_tablet_manager.cc:1888] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:59.724108 26658 log.cc:1196] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/d6dfeeb6b16a4f92bc756bf8933c7fb1
I20240502 07:58:59.724413 26658 ts_tablet_manager.cc:1909] T d6dfeeb6b16a4f92bc756bf8933c7fb1 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:58:59.725136 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet d6dfeeb6b16a4f92bc756bf8933c7fb1 (table rebalance_test_table_14 [id=6fc58da1a2c74e7a90d58e172a67ce50]) successfully deleted
I20240502 07:58:59.745462 26230 raft_consensus.cc:482] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:59.745606 26230 raft_consensus.cc:504] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.746033 26230 leader_election.cc:288] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:59.746624 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7828cb273c64934853b2cd38368402f" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:59.746752 22372 raft_consensus.cc:2454] T f7828cb273c64934853b2cd38368402f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:58:59.747020 25020 leader_election.cc:302] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:58:59.747182 26230 raft_consensus.cc:2781] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:59.747234 26230 raft_consensus.cc:482] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:59.747285 26230 raft_consensus.cc:3037] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.748106 26230 raft_consensus.cc:504] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.748423 26230 leader_election.cc:288] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:58:59.748726 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7828cb273c64934853b2cd38368402f" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:59.748842 22372 raft_consensus.cc:3037] T f7828cb273c64934853b2cd38368402f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.749912 22372 raft_consensus.cc:2454] T f7828cb273c64934853b2cd38368402f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:58:59.750447 25020 leader_election.cc:302] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:58:59.750646 26230 raft_consensus.cc:2781] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:59.750722 26230 raft_consensus.cc:686] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:58:59.750850 26230 consensus_queue.cc:237] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.752542 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7828cb273c64934853b2cd38368402f" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:58:59.752699 24951 raft_consensus.cc:2454] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:58:59.752609 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7828cb273c64934853b2cd38368402f" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:58:59.753063 24952 raft_consensus.cc:3037] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.754012 22223 catalog_manager.cc:5484] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:59.754127 24952 raft_consensus.cc:2454] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:58:59.760128 22372 consensus_queue.cc:237] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:58:59.761560 24820 raft_consensus.cc:1261] T c2422c666baa48078030c43d1c0938f9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:58:59.761585 22225 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:58:59.761941 26322 consensus_queue.cc:1035] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:59.762480 22765 raft_consensus.cc:1261] T c2422c666baa48078030c43d1c0938f9 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:58:59.763455 26322 consensus_queue.cc:1035] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:58:59.763983 26558 raft_consensus.cc:2932] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:59.766360 24821 raft_consensus.cc:2932] T c2422c666baa48078030c43d1c0938f9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:59.768173 26579 tablet.cc:1722] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:58:59.768672 26581 tablet.cc:1722] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:58:59.768702 22765 raft_consensus.cc:2932] T c2422c666baa48078030c43d1c0938f9 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:59.770973 22223 catalog_manager.cc:5484] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:58:59.772722 26580 tablet.cc:1722] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:58:59.774080 22229 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=725e9a9b5c984fe5a52ece225d99732b] alter complete (version 3)
I20240502 07:58:59.775511 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet c2422c666baa48078030c43d1c0938f9 with delete_type TABLET_DATA_TOMBSTONED (TS e51b10af49484aaa8f0267aebc1bf27f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47186
I20240502 07:58:59.775758 26659 tablet_replica.cc:330] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:58:59.775892 26659 raft_consensus.cc:2229] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.776043 26659 raft_consensus.cc:2258] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.776738 26659 ts_tablet_manager.cc:1875] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:58:59.779093 26659 ts_tablet_manager.cc:1888] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:58:59.779455 26659 log.cc:1196] T c2422c666baa48078030c43d1c0938f9 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/c2422c666baa48078030c43d1c0938f9
I20240502 07:58:59.779984 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet c2422c666baa48078030c43d1c0938f9 (table rebalance_test_table_19 [id=4e87626760fa4d8098e55dfa6d1c55a9]) successfully deleted
I20240502 07:58:59.816859 25429 raft_consensus.cc:482] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:59.816982 25429 raft_consensus.cc:504] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:59.817399 25429 leader_election.cc:288] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:59.818042 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c672cd17f991499b92271126e9cf9cb6" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:58:59.818181 24952 raft_consensus.cc:2454] T c672cd17f991499b92271126e9cf9cb6 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:58:59.818545 25151 leader_election.cc:302] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:58:59.818758 25429 raft_consensus.cc:2781] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:59.818815 25429 raft_consensus.cc:482] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:59.818852 25429 raft_consensus.cc:3037] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.819041 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c672cd17f991499b92271126e9cf9cb6" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:58:59.819161 22765 raft_consensus.cc:2454] T c672cd17f991499b92271126e9cf9cb6 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:58:59.819823 25429 raft_consensus.cc:504] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:59.820125 25429 leader_election.cc:288] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:58:59.821285 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c672cd17f991499b92271126e9cf9cb6" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:58:59.821444 24952 raft_consensus.cc:3037] T c672cd17f991499b92271126e9cf9cb6 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.822320 24952 raft_consensus.cc:2454] T c672cd17f991499b92271126e9cf9cb6 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:58:59.822700 22765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c672cd17f991499b92271126e9cf9cb6" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:58:59.822813 22765 raft_consensus.cc:3037] T c672cd17f991499b92271126e9cf9cb6 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.823535 22765 raft_consensus.cc:2454] T c672cd17f991499b92271126e9cf9cb6 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:58:59.823740 25151 leader_election.cc:302] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:58:59.823909 25429 raft_consensus.cc:2781] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:59.824077 25429 raft_consensus.cc:686] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:58:59.824230 25429 consensus_queue.cc:237] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:58:59.825960 22224 catalog_manager.cc:5484] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:58:59.878875 22222 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:58:59.879014 22222 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:58:59.880973 22222 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 9143376336e94ed9b1c75ddbeaf71034
I20240502 07:58:59.881597 24800 tablet_service.cc:1514] Processing DeleteTablet for tablet 9143376336e94ed9b1c75ddbeaf71034 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:58:59.881614 25195 tablet_service.cc:1514] Processing DeleteTablet for tablet 9143376336e94ed9b1c75ddbeaf71034 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:58:59.881609 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet 9143376336e94ed9b1c75ddbeaf71034 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:58:59 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:58:59.881799 26658 tablet_replica.cc:330] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:58:59.881938 26658 raft_consensus.cc:2229] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.881982 26723 tablet_replica.cc:330] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:58:59.882015 26722 tablet_replica.cc:330] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:58:59.882084 26658 raft_consensus.cc:2258] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.882162 26722 raft_consensus.cc:2229] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:58:59.882162 26723 raft_consensus.cc:2229] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:58:59.882308 26723 raft_consensus.cc:2258] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.882412 26722 raft_consensus.cc:2258] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:58:59.882733 26658 ts_tablet_manager.cc:1875] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.882977 26723 ts_tablet_manager.cc:1875] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.882977 26722 ts_tablet_manager.cc:1875] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:58:59.884871 26658 ts_tablet_manager.cc:1888] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:59.884949 26658 log.cc:1196] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/9143376336e94ed9b1c75ddbeaf71034
I20240502 07:58:59.885079 26723 ts_tablet_manager.cc:1888] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:59.885079 26722 ts_tablet_manager.cc:1888] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:58:59.885143 26723 log.cc:1196] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/9143376336e94ed9b1c75ddbeaf71034
I20240502 07:58:59.885143 26722 log.cc:1196] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/9143376336e94ed9b1c75ddbeaf71034
I20240502 07:58:59.885205 26658 ts_tablet_manager.cc:1909] T 9143376336e94ed9b1c75ddbeaf71034 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:58:59.885449 26722 ts_tablet_manager.cc:1909] T 9143376336e94ed9b1c75ddbeaf71034 P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:58:59.885483 26723 ts_tablet_manager.cc:1909] T 9143376336e94ed9b1c75ddbeaf71034 P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:58:59.885897 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet 9143376336e94ed9b1c75ddbeaf71034 (table rebalancer_dynamic_table_new_name [id=725e9a9b5c984fe5a52ece225d99732b]) successfully deleted
I20240502 07:58:59.886282 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet 9143376336e94ed9b1c75ddbeaf71034 (table rebalancer_dynamic_table_new_name [id=725e9a9b5c984fe5a52ece225d99732b]) successfully deleted
I20240502 07:58:59.886703 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 9143376336e94ed9b1c75ddbeaf71034 (table rebalancer_dynamic_table_new_name [id=725e9a9b5c984fe5a52ece225d99732b]) successfully deleted
I20240502 07:58:59.958737 26230 raft_consensus.cc:482] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:58:59.958863 26230 raft_consensus.cc:504] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.959264 26230 leader_election.cc:288] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:58:59.959753 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f44c905bfaa4db093a1b39b6c21e03c" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:58:59.959779 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f44c905bfaa4db093a1b39b6c21e03c" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:58:59.959903 24952 raft_consensus.cc:2454] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:58:59.959903 22372 raft_consensus.cc:2454] T 9f44c905bfaa4db093a1b39b6c21e03c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:58:59.960197 25020 leader_election.cc:302] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:58:59.960356 26230 raft_consensus.cc:2781] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:58:59.960409 26230 raft_consensus.cc:482] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:58:59.960445 26230 raft_consensus.cc:3037] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.961256 26230 raft_consensus.cc:504] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.961586 26230 leader_election.cc:288] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:58:59.961885 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f44c905bfaa4db093a1b39b6c21e03c" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:58:59.961956 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f44c905bfaa4db093a1b39b6c21e03c" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:58:59.962010 22372 raft_consensus.cc:3037] T 9f44c905bfaa4db093a1b39b6c21e03c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.962062 24952 raft_consensus.cc:3037] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:58:59.962986 24952 raft_consensus.cc:2454] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:58:59.962986 22372 raft_consensus.cc:2454] T 9f44c905bfaa4db093a1b39b6c21e03c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:58:59.963431 25020 leader_election.cc:302] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
I20240502 07:58:59.963583 26230 raft_consensus.cc:2781] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:58:59.963671 26230 raft_consensus.cc:686] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:58:59.963809 26230 consensus_queue.cc:237] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:58:59.965570 22222 catalog_manager.cc:5484] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:58:59.979444 24932 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:59.983945 22222 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:58:59.987175 22351 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:58:59.989205 22612 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:58:59.993856 22745 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:58:59.994261 24800 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:58:59.997962 25063 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:58:59.998553 25195 tablet_service.cc:1467] Processing CreateTablet for tablet 059b3e524b574be183ef8d6d16522202 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9e4522ab76fc47e2bb4dd432ae621e3a]), partition=
I20240502 07:58:59.998848 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 059b3e524b574be183ef8d6d16522202. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:58:59.998972 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 059b3e524b574be183ef8d6d16522202 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9e4522ab76fc47e2bb4dd432ae621e3a]), partition=
I20240502 07:58:59.999200 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 059b3e524b574be183ef8d6d16522202. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.001220 26626 tablet_bootstrap.cc:492] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:00.001188 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 059b3e524b574be183ef8d6d16522202 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9e4522ab76fc47e2bb4dd432ae621e3a]), partition=
I20240502 07:59:00.001461 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 059b3e524b574be183ef8d6d16522202. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.002156 26626 tablet_bootstrap.cc:654] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.003904 25194 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:00.004029 26413 tablet_bootstrap.cc:492] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:00.004576 26626 tablet_bootstrap.cc:492] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:00.004659 26626 ts_tablet_manager.cc:1367] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:00.004952 26413 tablet_bootstrap.cc:654] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.005141 26626 raft_consensus.cc:348] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:00.005259 26626 raft_consensus.cc:374] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.005299 26626 raft_consensus.cc:729] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.005462 26626 consensus_queue.cc:260] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:00.006461 26626 ts_tablet_manager.cc:1398] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:00.008822 26737 tablet_bootstrap.cc:492] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:00.011274 26737 tablet_bootstrap.cc:654] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.024976 26413 tablet_bootstrap.cc:492] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:00.025085 26413 ts_tablet_manager.cc:1367] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 07:59:00.025597 26413 raft_consensus.cc:348] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:00.025717 26413 raft_consensus.cc:374] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.025744 26413 raft_consensus.cc:729] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.025859 26413 consensus_queue.cc:260] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:00.026096 26413 ts_tablet_manager.cc:1398] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:00.032245 22483 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:00.050026 26737 tablet_bootstrap.cc:492] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:00.050148 26737 ts_tablet_manager.cc:1367] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.000s
I20240502 07:59:00.050755 26737 raft_consensus.cc:348] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:00.050853 26737 raft_consensus.cc:374] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.050874 26737 raft_consensus.cc:729] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.050992 26737 consensus_queue.cc:260] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:00.051334 26737 ts_tablet_manager.cc:1398] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:00.097096 22764 consensus_queue.cc:237] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.097733 26225 raft_consensus.cc:482] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:00.097831 26225 raft_consensus.cc:504] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:00.098222 26225 leader_election.cc:288] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:00.098770 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "059b3e524b574be183ef8d6d16522202" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:00.098801 22502 raft_consensus.cc:1261] T 92162f8f5fcb4baf96fa8004b7b41b9b P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:00.098901 24821 raft_consensus.cc:2454] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:00.099444 24887 leader_election.cc:302] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:00.099591 25083 raft_consensus.cc:1261] T 92162f8f5fcb4baf96fa8004b7b41b9b P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:00.099664 26225 raft_consensus.cc:2781] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:00.099716 26225 raft_consensus.cc:482] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:00.099742 26225 raft_consensus.cc:3037] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.099892 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "059b3e524b574be183ef8d6d16522202" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:00.100013 25214 raft_consensus.cc:2454] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:00.100080 24452 consensus_queue.cc:1035] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:00.100850 24452 consensus_queue.cc:1035] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:00.100886 26225 raft_consensus.cc:504] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:00.101243 26225 leader_election.cc:288] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:00.101662 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "059b3e524b574be183ef8d6d16522202" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:00.101773 25214 raft_consensus.cc:3037] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.102759 25214 raft_consensus.cc:2454] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:00.103245 24888 leader_election.cc:302] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:00.103861 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "059b3e524b574be183ef8d6d16522202" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:00.103981 24821 raft_consensus.cc:3037] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.103978 26602 raft_consensus.cc:2932] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:00.104883 24821 raft_consensus.cc:2454] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:00.105675 26225 raft_consensus.cc:2781] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:00.105818 26225 raft_consensus.cc:686] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:00.105952 26225 consensus_queue.cc:237] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:00.107566 22222 catalog_manager.cc:5484] T 92162f8f5fcb4baf96fa8004b7b41b9b P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:00.109189 22224 catalog_manager.cc:5484] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:00.110086 25083 raft_consensus.cc:2932] T 92162f8f5fcb4baf96fa8004b7b41b9b P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:00.112808 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet 92162f8f5fcb4baf96fa8004b7b41b9b with delete_type TABLET_DATA_TOMBSTONED (TS e51b10af49484aaa8f0267aebc1bf27f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:00.113075 26659 tablet_replica.cc:330] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:00.113265 26659 raft_consensus.cc:2229] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.113260 22502 raft_consensus.cc:2932] T 92162f8f5fcb4baf96fa8004b7b41b9b P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:00.113449 26659 raft_consensus.cc:2258] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.113915 26659 ts_tablet_manager.cc:1875] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:00.115994 26659 ts_tablet_manager.cc:1888] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:00.116047 26659 log.cc:1196] T 92162f8f5fcb4baf96fa8004b7b41b9b P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/92162f8f5fcb4baf96fa8004b7b41b9b
I20240502 07:59:00.116659 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 92162f8f5fcb4baf96fa8004b7b41b9b (table rebalance_test_table_7 [id=255d8aedf8f845a1b8080105bc5b3a93]) successfully deleted
I20240502 07:59:00.192703 22223 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_16" } modify_external_catalogs: true
I20240502 07:59:00.192864 22223 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_16" } modify_external_catalogs: true
I20240502 07:59:00.193024 24932 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:00.196589 22223 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet ca6b1845b2f8405da3ffa3fcb9d86150
I20240502 07:59:00.196935 22223 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 9dddf3cd3f104977b369e787913af039
I20240502 07:59:00.197160 22223 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 4 replicas of tablet fa14f8f327cb4221aabeedac16d258b6
I20240502 07:59:00.197191 24800 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:00.197387 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet ca6b1845b2f8405da3ffa3fcb9d86150 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:00.197584 26075 tablet_replica.cc:330] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:00.197742 26075 raft_consensus.cc:2229] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:00.197983 26075 raft_consensus.cc:2258] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.198590 26075 ts_tablet_manager.cc:1875] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.201174 26075 ts_tablet_manager.cc:1888] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:59:00.201241 26075 log.cc:1196] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/ca6b1845b2f8405da3ffa3fcb9d86150
I20240502 07:59:00.202605 22482 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dddf3cd3f104977b369e787913af039 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:00.202673 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dddf3cd3f104977b369e787913af039 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:00.202836 22614 tablet_service.cc:1514] Processing DeleteTablet for tablet fa14f8f327cb4221aabeedac16d258b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:00.202880 25812 tablet_replica.cc:330] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:00.202883 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet ca6b1845b2f8405da3ffa3fcb9d86150 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:00.202889 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet ca6b1845b2f8405da3ffa3fcb9d86150 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:00.202998 22744 tablet_service.cc:1514] Processing DeleteTablet for tablet fa14f8f327cb4221aabeedac16d258b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:00.203017 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dddf3cd3f104977b369e787913af039 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:00.203037 25812 raft_consensus.cc:2229] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:00.203088 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet fa14f8f327cb4221aabeedac16d258b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:00.203121 22350 tablet_service.cc:1514] Processing DeleteTablet for tablet fa14f8f327cb4221aabeedac16d258b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:00.203281 25812 raft_consensus.cc:2258] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.203861 25812 ts_tablet_manager.cc:1875] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.203974 26722 tablet_replica.cc:330] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:00.204145 26722 raft_consensus.cc:2229] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.204265 26722 raft_consensus.cc:2258] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.204855 26722 ts_tablet_manager.cc:1875] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.206270 25812 ts_tablet_manager.cc:1888] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:59:00.206351 25812 log.cc:1196] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/9dddf3cd3f104977b369e787913af039
I20240502 07:59:00.206622 25812 ts_tablet_manager.cc:1909] T 9dddf3cd3f104977b369e787913af039 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:59:00.206764 26659 tablet_replica.cc:330] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:00.206897 26659 raft_consensus.cc:2229] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.207033 26659 raft_consensus.cc:2258] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.207198 26722 ts_tablet_manager.cc:1888] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:00.207252 26722 log.cc:1196] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/fa14f8f327cb4221aabeedac16d258b6
I20240502 07:59:00.207494 26722 ts_tablet_manager.cc:1909] T fa14f8f327cb4221aabeedac16d258b6 P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:59:00.207530 26659 ts_tablet_manager.cc:1875] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.208273 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet fa14f8f327cb4221aabeedac16d258b6 (table rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]) successfully deleted
I20240502 07:59:00.210630 26075 ts_tablet_manager.cc:1909] T ca6b1845b2f8405da3ffa3fcb9d86150 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:59:00.210852 25063 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:00.211378 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet ca6b1845b2f8405da3ffa3fcb9d86150 (table rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]) successfully deleted
I20240502 07:59:00.211829 26075 tablet_replica.cc:330] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:00.212204 26075 raft_consensus.cc:2229] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.212293 26811 tablet_replica.cc:330] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:00.212273 26659 ts_tablet_manager.cc:1888] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:00.212352 26075 raft_consensus.cc:2258] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.212369 26659 log.cc:1196] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/fa14f8f327cb4221aabeedac16d258b6
I20240502 07:59:00.212493 26811 raft_consensus.cc:2229] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.212646 26659 ts_tablet_manager.cc:1909] T fa14f8f327cb4221aabeedac16d258b6 P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:59:00.212641 26811 raft_consensus.cc:2258] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.213042 26075 ts_tablet_manager.cc:1875] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.213235 26659 tablet_replica.cc:330] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:00.213282 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet fa14f8f327cb4221aabeedac16d258b6 (table rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]) successfully deleted
I20240502 07:59:00.213382 26659 raft_consensus.cc:2229] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.213507 26659 raft_consensus.cc:2258] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.213569 26811 ts_tablet_manager.cc:1875] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.213958 26659 ts_tablet_manager.cc:1875] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.215428 26075 ts_tablet_manager.cc:1888] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:59:00.215488 26075 log.cc:1196] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/9dddf3cd3f104977b369e787913af039
I20240502 07:59:00.215749 26075 ts_tablet_manager.cc:1909] T 9dddf3cd3f104977b369e787913af039 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:59:00.215907 26659 ts_tablet_manager.cc:1888] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:59:00.215907 26811 ts_tablet_manager.cc:1888] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:59:00.215970 26659 log.cc:1196] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/ca6b1845b2f8405da3ffa3fcb9d86150
I20240502 07:59:00.215971 26811 log.cc:1196] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/9dddf3cd3f104977b369e787913af039
I20240502 07:59:00.216224 26811 ts_tablet_manager.cc:1909] T 9dddf3cd3f104977b369e787913af039 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:59:00.217032 26811 tablet_replica.cc:330] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:00.217195 26811 raft_consensus.cc:2229] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.217581 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 9dddf3cd3f104977b369e787913af039 (table rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]) successfully deleted
I20240502 07:59:00.217823 26075 tablet_replica.cc:330] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:00.217984 26075 raft_consensus.cc:2229] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.218129 26075 raft_consensus.cc:2258] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.218767 26075 ts_tablet_manager.cc:1875] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.219900 22223 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:00.225150 26811 raft_consensus.cc:2258] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.225411 22612 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:00.225678 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 9dddf3cd3f104977b369e787913af039 (table rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]) successfully deleted
I20240502 07:59:00.226423 26811 ts_tablet_manager.cc:1875] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.228888 26075 ts_tablet_manager.cc:1888] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:00.228989 26075 log.cc:1196] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/fa14f8f327cb4221aabeedac16d258b6
I20240502 07:59:00.229271 26075 ts_tablet_manager.cc:1909] T fa14f8f327cb4221aabeedac16d258b6 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:59:00.230032 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet fa14f8f327cb4221aabeedac16d258b6 (table rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]) successfully deleted
I20240502 07:59:00.231884 24821 raft_consensus.cc:1261] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:00.231879 25214 raft_consensus.cc:1261] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:00.232318 26811 ts_tablet_manager.cc:1888] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:59:00.232393 26811 log.cc:1196] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/ca6b1845b2f8405da3ffa3fcb9d86150
I20240502 07:59:00.232342 26758 consensus_queue.cc:1035] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.232642 26811 ts_tablet_manager.cc:1909] T ca6b1845b2f8405da3ffa3fcb9d86150 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:59:00.232775 26225 consensus_queue.cc:1035] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.234113 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet ca6b1845b2f8405da3ffa3fcb9d86150 (table rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]) successfully deleted
I20240502 07:59:00.235536 22223 catalog_manager.cc:5576] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661) does not have the latest schema for tablet 059b3e524b574be183ef8d6d16522202 (table rebalancer_dynamic_table [id=9e4522ab76fc47e2bb4dd432ae621e3a]). Expected version 1 got 0
I20240502 07:59:00.236896 22222 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 059b3e524b574be183ef8d6d16522202 (table rebalancer_dynamic_table [id=9e4522ab76fc47e2bb4dd432ae621e3a]). Expected version 1 got 0
I20240502 07:59:00.238384 22483 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:00.238575 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 9dddf3cd3f104977b369e787913af039 (table rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]) successfully deleted
I20240502 07:59:00.238799 26581 tablet.cc:1722] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:00.238869 25812 tablet_replica.cc:330] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:00.238991 25812 raft_consensus.cc:2229] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:00.239096 26581 tablet.cc:1620] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:00.239485 26815 tablet.cc:1722] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:00.239696 26580 tablet.cc:1722] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:00.239826 25812 raft_consensus.cc:2258] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.239979 26580 tablet.cc:1620] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:00.240325 26815 tablet.cc:1620] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:00.240492 25812 ts_tablet_manager.cc:1875] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.240720 22350 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:00.242831 25195 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:00.244956 22222 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=9e4522ab76fc47e2bb4dd432ae621e3a] alter complete (version 1)
I20240502 07:59:00.261822 25812 ts_tablet_manager.cc:1888] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:00.261921 25812 log.cc:1196] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/fa14f8f327cb4221aabeedac16d258b6
I20240502 07:59:00.262173 25812 ts_tablet_manager.cc:1909] T fa14f8f327cb4221aabeedac16d258b6 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:59:00.262499 22744 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:00.262730 26659 ts_tablet_manager.cc:1909] T ca6b1845b2f8405da3ffa3fcb9d86150 P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:59:00.263530 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet ca6b1845b2f8405da3ffa3fcb9d86150 (table rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]) successfully deleted
I20240502 07:59:00.265002 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet fa14f8f327cb4221aabeedac16d258b6 (table rebalance_test_table_16 [id=c09f49e6f05e4cab88a126bdd611a5f2]) successfully deleted
I20240502 07:59:00.301980 26230 consensus_queue.cc:1035] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.306728 26230 consensus_queue.cc:1035] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.317494 25429 consensus_queue.cc:1035] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.321455 25429 consensus_queue.cc:1035] T c672cd17f991499b92271126e9cf9cb6 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.326066 22633 consensus_queue.cc:237] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:00.327951 22765 raft_consensus.cc:1261] T 648a25ebebe44061bca2844ec5b048c5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:00.328234 24821 raft_consensus.cc:1261] T 648a25ebebe44061bca2844ec5b048c5 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:00.328461 26771 consensus_queue.cc:1035] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:00.329031 26771 consensus_queue.cc:1035] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:00.330340 25587 raft_consensus.cc:2932] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:00.330790 22765 raft_consensus.cc:2932] T 648a25ebebe44061bca2844ec5b048c5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:00.331271 24821 raft_consensus.cc:2932] T 648a25ebebe44061bca2844ec5b048c5 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:00.333792 22222 catalog_manager.cc:5484] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:00.336792 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet 648a25ebebe44061bca2844ec5b048c5 with delete_type TABLET_DATA_TOMBSTONED (TS fabddb16c85944648a710ec52fc7244c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:00.337085 26811 tablet_replica.cc:330] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:00.337231 26811 raft_consensus.cc:2229] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.337425 26811 raft_consensus.cc:2258] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.337986 26811 ts_tablet_manager.cc:1875] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:00.340123 26811 ts_tablet_manager.cc:1888] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:00.340175 26811 log.cc:1196] T 648a25ebebe44061bca2844ec5b048c5 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/648a25ebebe44061bca2844ec5b048c5
I20240502 07:59:00.340693 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 648a25ebebe44061bca2844ec5b048c5 (table rebalance_test_table_5 [id=80b9c0f932754699a4f41d2c2188b70b]) successfully deleted
I20240502 07:59:00.348206 22229 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:00.354657 26815 tablet.cc:1722] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:00.354991 26815 tablet.cc:1620] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:00.355429 26581 tablet.cc:1722] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:00.355697 26581 tablet.cc:1620] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:00.355964 26580 tablet.cc:1722] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:00.356240 26580 tablet.cc:1620] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:00.361440 22223 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=9e4522ab76fc47e2bb4dd432ae621e3a] alter complete (version 2)
I20240502 07:59:00.391340 26230 consensus_queue.cc:1035] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.400382 26230 consensus_queue.cc:1035] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.423264 24800 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:00.423862 25063 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:00.428706 22350 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:00.431773 24932 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:00.444099 22612 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:00.444180 25195 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:00.448104 22483 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:00.463459 22225 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:00.470000 22744 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:00.473088 26815 tablet.cc:1722] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:00.473606 26580 tablet.cc:1722] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:00.479911 22230 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=9e4522ab76fc47e2bb4dd432ae621e3a] alter complete (version 3)
I20240502 07:59:00.480304 26581 tablet.cc:1722] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:00.494385 22225 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:00.500598 24800 tablet_service.cc:1467] Processing CreateTablet for tablet e3fd481f3c78497eb629b0180c07674f (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=03e205afe2d642629a8ba4c48516b9d5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.500916 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3fd481f3c78497eb629b0180c07674f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.501502 25194 tablet_service.cc:1467] Processing CreateTablet for tablet f49fcc586e6f498e93723d97b99c077e (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=03e205afe2d642629a8ba4c48516b9d5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.501797 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f49fcc586e6f498e93723d97b99c077e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.501942 24932 tablet_service.cc:1467] Processing CreateTablet for tablet cba71d2a28984e84b1950d59ec2e5a6a (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=03e205afe2d642629a8ba4c48516b9d5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.502192 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cba71d2a28984e84b1950d59ec2e5a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.503063 26737 tablet_bootstrap.cc:492] T e3fd481f3c78497eb629b0180c07674f P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:00.503891 26737 tablet_bootstrap.cc:654] T e3fd481f3c78497eb629b0180c07674f P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.503960 26413 tablet_bootstrap.cc:492] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:00.501502 25195 tablet_service.cc:1467] Processing CreateTablet for tablet e3fd481f3c78497eb629b0180c07674f (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=03e205afe2d642629a8ba4c48516b9d5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.504498 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3fd481f3c78497eb629b0180c07674f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.504683 26626 tablet_bootstrap.cc:492] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:00.505162 26413 tablet_bootstrap.cc:654] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.505611 26626 tablet_bootstrap.cc:654] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.506582 26413 tablet_bootstrap.cc:492] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:00.506533 24800 tablet_service.cc:1467] Processing CreateTablet for tablet cba71d2a28984e84b1950d59ec2e5a6a (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=03e205afe2d642629a8ba4c48516b9d5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.506671 26413 ts_tablet_manager.cc:1367] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:00.506805 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cba71d2a28984e84b1950d59ec2e5a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.507150 26413 raft_consensus.cc:348] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.507249 26413 raft_consensus.cc:374] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.507277 26413 raft_consensus.cc:729] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.507350 22483 tablet_service.cc:1467] Processing CreateTablet for tablet f49fcc586e6f498e93723d97b99c077e (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=03e205afe2d642629a8ba4c48516b9d5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.507350 25062 tablet_service.cc:1467] Processing CreateTablet for tablet f49fcc586e6f498e93723d97b99c077e (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=03e205afe2d642629a8ba4c48516b9d5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.507404 26413 consensus_queue.cc:260] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.507612 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f49fcc586e6f498e93723d97b99c077e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.507609 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f49fcc586e6f498e93723d97b99c077e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.508656 26413 ts_tablet_manager.cc:1398] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:00.508877 26413 tablet_bootstrap.cc:492] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:00.509601 25063 tablet_service.cc:1467] Processing CreateTablet for tablet e3fd481f3c78497eb629b0180c07674f (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=03e205afe2d642629a8ba4c48516b9d5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.509848 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3fd481f3c78497eb629b0180c07674f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.509936 26413 tablet_bootstrap.cc:654] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.511847 25064 tablet_service.cc:1467] Processing CreateTablet for tablet cba71d2a28984e84b1950d59ec2e5a6a (DEFAULT_TABLE table=rebalancer_extra_table_10 [id=03e205afe2d642629a8ba4c48516b9d5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:00.512043 25064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cba71d2a28984e84b1950d59ec2e5a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.516603 26896 tablet_bootstrap.cc:492] T f49fcc586e6f498e93723d97b99c077e P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:00.517378 26413 tablet_bootstrap.cc:492] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:00.517530 26413 ts_tablet_manager.cc:1367] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20240502 07:59:00.517959 26896 tablet_bootstrap.cc:654] T f49fcc586e6f498e93723d97b99c077e P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.518038 26413 raft_consensus.cc:348] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.518220 26413 raft_consensus.cc:374] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.518296 26413 raft_consensus.cc:729] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.518468 26413 consensus_queue.cc:260] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.518934 26413 ts_tablet_manager.cc:1398] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:00.519255 26899 tablet_bootstrap.cc:492] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:00.519556 26626 tablet_bootstrap.cc:492] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:00.519649 26626 ts_tablet_manager.cc:1367] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 07:59:00.520105 26626 raft_consensus.cc:348] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:00.520206 26626 raft_consensus.cc:374] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.520242 26626 raft_consensus.cc:729] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.520361 26899 tablet_bootstrap.cc:654] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.520395 26626 consensus_queue.cc:260] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:00.520468 26737 tablet_bootstrap.cc:492] T e3fd481f3c78497eb629b0180c07674f P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:00.520553 26737 ts_tablet_manager.cc:1367] T e3fd481f3c78497eb629b0180c07674f P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 07:59:00.520689 26626 ts_tablet_manager.cc:1398] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.520978 26737 raft_consensus.cc:348] T e3fd481f3c78497eb629b0180c07674f P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.521080 26737 raft_consensus.cc:374] T e3fd481f3c78497eb629b0180c07674f P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.521111 26737 raft_consensus.cc:729] T e3fd481f3c78497eb629b0180c07674f P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.521232 26737 consensus_queue.cc:260] T e3fd481f3c78497eb629b0180c07674f P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.521524 26737 ts_tablet_manager.cc:1398] T e3fd481f3c78497eb629b0180c07674f P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.521404 26899 tablet_bootstrap.cc:492] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:00.521649 26899 ts_tablet_manager.cc:1367] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:00.521687 26737 tablet_bootstrap.cc:492] T cba71d2a28984e84b1950d59ec2e5a6a P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:00.522246 26899 raft_consensus.cc:348] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.522351 26899 raft_consensus.cc:374] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.522382 26899 raft_consensus.cc:729] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.522497 26737 tablet_bootstrap.cc:654] T cba71d2a28984e84b1950d59ec2e5a6a P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.522490 26899 consensus_queue.cc:260] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.522761 26899 ts_tablet_manager.cc:1398] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.528656 26896 tablet_bootstrap.cc:492] T f49fcc586e6f498e93723d97b99c077e P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:00.528764 26896 ts_tablet_manager.cc:1367] T f49fcc586e6f498e93723d97b99c077e P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 07:59:00.529196 26896 raft_consensus.cc:348] T f49fcc586e6f498e93723d97b99c077e P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.529306 26896 raft_consensus.cc:374] T f49fcc586e6f498e93723d97b99c077e P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.529356 26896 raft_consensus.cc:729] T f49fcc586e6f498e93723d97b99c077e P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.529484 26896 consensus_queue.cc:260] T f49fcc586e6f498e93723d97b99c077e P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.529564 26737 tablet_bootstrap.cc:492] T cba71d2a28984e84b1950d59ec2e5a6a P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:00.529661 26737 ts_tablet_manager.cc:1367] T cba71d2a28984e84b1950d59ec2e5a6a P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:59:00.529738 26896 ts_tablet_manager.cc:1398] T f49fcc586e6f498e93723d97b99c077e P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:00.529920 26896 tablet_bootstrap.cc:492] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:00.530056 26737 raft_consensus.cc:348] T cba71d2a28984e84b1950d59ec2e5a6a P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:00.530154 26737 raft_consensus.cc:374] T cba71d2a28984e84b1950d59ec2e5a6a P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.530187 26737 raft_consensus.cc:729] T cba71d2a28984e84b1950d59ec2e5a6a P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.530298 26737 consensus_queue.cc:260] T cba71d2a28984e84b1950d59ec2e5a6a P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:00.530551 26737 ts_tablet_manager.cc:1398] T cba71d2a28984e84b1950d59ec2e5a6a P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.530843 26896 tablet_bootstrap.cc:654] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.534584 26758 raft_consensus.cc:482] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:00.534710 26758 raft_consensus.cc:504] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:00.535138 26758 leader_election.cc:288] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:00.535418 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cba71d2a28984e84b1950d59ec2e5a6a" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:00.535812 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cba71d2a28984e84b1950d59ec2e5a6a" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:00.535935 24821 raft_consensus.cc:2454] T cba71d2a28984e84b1950d59ec2e5a6a P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
W20240502 07:59:00.535941 24885 leader_election.cc:341] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): Illegal state: must be running to vote when last-logged opid is not known
I20240502 07:59:00.536458 24887 leader_election.cc:302] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1aecef81733c47b3b468504934357231, 4b4e7eeb7f3f42119c0363a79febc54b; no voters: 227eec4b5f014d349661f6e0b39b9055
I20240502 07:59:00.536852 26896 tablet_bootstrap.cc:492] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:00.536935 26896 ts_tablet_manager.cc:1367] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240502 07:59:00.537382 26758 raft_consensus.cc:2781] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:00.537438 26758 raft_consensus.cc:482] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:00.537477 26758 raft_consensus.cc:3037] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.537453 26896 raft_consensus.cc:348] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.537567 26896 raft_consensus.cc:374] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.537603 26896 raft_consensus.cc:729] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.537740 26896 consensus_queue.cc:260] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.537990 26896 ts_tablet_manager.cc:1398] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.538125 26896 tablet_bootstrap.cc:492] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:00.538401 26758 raft_consensus.cc:504] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:00.538702 26758 leader_election.cc:288] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:00.538988 26896 tablet_bootstrap.cc:654] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.538959 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cba71d2a28984e84b1950d59ec2e5a6a" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:00.539052 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cba71d2a28984e84b1950d59ec2e5a6a" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:00.539155 24821 raft_consensus.cc:3037] T cba71d2a28984e84b1950d59ec2e5a6a P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
W20240502 07:59:00.539274 24885 leader_election.cc:341] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): Illegal state: must be running to vote when last-logged opid is not known
I20240502 07:59:00.540004 26896 tablet_bootstrap.cc:492] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:00.540081 26896 ts_tablet_manager.cc:1367] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:00.540158 24821 raft_consensus.cc:2454] T cba71d2a28984e84b1950d59ec2e5a6a P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:00.540534 26896 raft_consensus.cc:348] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:00.540712 26896 raft_consensus.cc:374] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.540796 26896 raft_consensus.cc:729] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.541124 26896 consensus_queue.cc:260] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:00.541566 26896 ts_tablet_manager.cc:1398] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.542116 24887 leader_election.cc:302] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1aecef81733c47b3b468504934357231, 4b4e7eeb7f3f42119c0363a79febc54b; no voters: 227eec4b5f014d349661f6e0b39b9055
I20240502 07:59:00.543838 26758 raft_consensus.cc:2781] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:00.543934 26758 raft_consensus.cc:686] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:00.544067 26758 consensus_queue.cc:237] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:00.545874 22222 catalog_manager.cc:5484] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:00.558202 22765 consensus_queue.cc:237] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:00.559612 24952 raft_consensus.cc:1261] T c5a30e1ebd7747eb911fba8437e42077 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:00.559670 22372 raft_consensus.cc:1261] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:00.560022 26602 consensus_queue.cc:1035] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:00.560225 24452 consensus_queue.cc:1035] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:00.562161 26602 raft_consensus.cc:2932] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:00.562610 22372 raft_consensus.cc:2932] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:00.562707 24952 raft_consensus.cc:2932] T c5a30e1ebd7747eb911fba8437e42077 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:00.565716 22230 catalog_manager.cc:5484] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:00.568470 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet c5a30e1ebd7747eb911fba8437e42077 with delete_type TABLET_DATA_TOMBSTONED (TS e51b10af49484aaa8f0267aebc1bf27f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:00.568732 26659 tablet_replica.cc:330] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:00.568908 26659 raft_consensus.cc:2229] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.569110 26659 raft_consensus.cc:2258] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.570077 26659 ts_tablet_manager.cc:1875] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:00.572358 26659 ts_tablet_manager.cc:1888] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:00.572450 26659 log.cc:1196] T c5a30e1ebd7747eb911fba8437e42077 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/c5a30e1ebd7747eb911fba8437e42077
I20240502 07:59:00.572973 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet c5a30e1ebd7747eb911fba8437e42077 (table rebalance_test_table_15 [id=4e81cbd084694ac8b1db3f5574dde601]) successfully deleted
I20240502 07:59:00.583935 22222 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:00.584101 22222 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:00.586382 22222 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 059b3e524b574be183ef8d6d16522202
I20240502 07:59:00.587021 24800 tablet_service.cc:1514] Processing DeleteTablet for tablet 059b3e524b574be183ef8d6d16522202 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:59:00.587074 25195 tablet_service.cc:1514] Processing DeleteTablet for tablet 059b3e524b574be183ef8d6d16522202 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:00.587260 24932 tablet_service.cc:1514] Processing DeleteTablet for tablet 059b3e524b574be183ef8d6d16522202 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:00.587371 26935 tablet_replica.cc:330] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:59:00.587494 26935 raft_consensus.cc:2229] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.587599 26937 tablet_replica.cc:330] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:00.587617 26935 raft_consensus.cc:2258] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.587742 26937 raft_consensus.cc:2229] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:00.587939 26937 raft_consensus.cc:2258] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.588320 26935 ts_tablet_manager.cc:1875] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.588472 26937 ts_tablet_manager.cc:1875] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.589519 26936 tablet_replica.cc:330] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:00.589731 26936 raft_consensus.cc:2229] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.589906 26936 raft_consensus.cc:2258] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.590538 26935 ts_tablet_manager.cc:1888] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:00.590546 26937 ts_tablet_manager.cc:1888] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:00.590595 26936 ts_tablet_manager.cc:1875] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.590633 26937 log.cc:1196] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/059b3e524b574be183ef8d6d16522202
I20240502 07:59:00.590696 26935 log.cc:1196] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/059b3e524b574be183ef8d6d16522202
I20240502 07:59:00.590931 26937 ts_tablet_manager.cc:1909] T 059b3e524b574be183ef8d6d16522202 P 1aecef81733c47b3b468504934357231: Deleting consensus metadata
I20240502 07:59:00.591035 26935 ts_tablet_manager.cc:1909] T 059b3e524b574be183ef8d6d16522202 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:59:00.591743 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet 059b3e524b574be183ef8d6d16522202 (table rebalancer_dynamic_table_new_name [id=9e4522ab76fc47e2bb4dd432ae621e3a]) successfully deleted
I20240502 07:59:00.591766 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet 059b3e524b574be183ef8d6d16522202 (table rebalancer_dynamic_table_new_name [id=9e4522ab76fc47e2bb4dd432ae621e3a]) successfully deleted
I20240502 07:59:00.593165 26936 ts_tablet_manager.cc:1888] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:00.593230 26936 log.cc:1196] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/059b3e524b574be183ef8d6d16522202
I20240502 07:59:00.593545 26936 ts_tablet_manager.cc:1909] T 059b3e524b574be183ef8d6d16522202 P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:59:00.594306 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 059b3e524b574be183ef8d6d16522202 (table rebalancer_dynamic_table_new_name [id=9e4522ab76fc47e2bb4dd432ae621e3a]) successfully deleted
I20240502 07:59:00.624751 26497 raft_consensus.cc:482] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:00.624884 26497 raft_consensus.cc:504] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.625277 26497 leader_election.cc:288] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:00.625686 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f49fcc586e6f498e93723d97b99c077e" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:00.625686 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f49fcc586e6f498e93723d97b99c077e" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:00.625828 25214 raft_consensus.cc:2454] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:00.625828 25083 raft_consensus.cc:2454] T f49fcc586e6f498e93723d97b99c077e P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:00.626183 22439 leader_election.cc:302] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:00.626623 26497 raft_consensus.cc:2781] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:00.626690 26497 raft_consensus.cc:482] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:00.626737 26497 raft_consensus.cc:3037] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.627727 26497 raft_consensus.cc:504] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.628142 26497 leader_election.cc:288] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:00.628549 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f49fcc586e6f498e93723d97b99c077e" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:00.628672 25083 raft_consensus.cc:3037] T f49fcc586e6f498e93723d97b99c077e P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.628803 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f49fcc586e6f498e93723d97b99c077e" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:00.628918 25214 raft_consensus.cc:3037] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.629753 25083 raft_consensus.cc:2454] T f49fcc586e6f498e93723d97b99c077e P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:00.630443 25214 raft_consensus.cc:2454] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:00.630784 22439 leader_election.cc:302] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:00.631395 26497 raft_consensus.cc:2781] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:00.631600 26497 raft_consensus.cc:686] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:59:00.631764 26497 consensus_queue.cc:237] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.634620 22230 catalog_manager.cc:5484] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:00.681824 25429 raft_consensus.cc:482] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:00.681886 26574 raft_consensus.cc:482] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:00.681953 25429 raft_consensus.cc:504] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.681978 26574 raft_consensus.cc:504] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:00.682365 26574 leader_election.cc:288] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:00.682435 25429 leader_election.cc:288] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:00.682842 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cba71d2a28984e84b1950d59ec2e5a6a" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:00.682961 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3fd481f3c78497eb629b0180c07674f" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:00.683104 24821 raft_consensus.cc:2454] T e3fd481f3c78497eb629b0180c07674f P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:00.683408 25150 leader_election.cc:302] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:00.683429 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cba71d2a28984e84b1950d59ec2e5a6a" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:00.683569 24820 raft_consensus.cc:2379] T cba71d2a28984e84b1950d59ec2e5a6a P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 227eec4b5f014d349661f6e0b39b9055 in current term 1: Already voted for candidate 1aecef81733c47b3b468504934357231 in this term.
I20240502 07:59:00.683617 25429 raft_consensus.cc:2781] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:00.683656 25429 raft_consensus.cc:482] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:00.683692 25429 raft_consensus.cc:3037] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.684235 25019 leader_election.cc:302] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055; no voters: 1aecef81733c47b3b468504934357231, 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:00.684387 26574 raft_consensus.cc:3037] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.684727 25429 raft_consensus.cc:504] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.685011 25429 leader_election.cc:288] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:00.685434 26574 raft_consensus.cc:2726] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 07:59:00.685592 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3fd481f3c78497eb629b0180c07674f" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:00.685696 24820 raft_consensus.cc:3037] T e3fd481f3c78497eb629b0180c07674f P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.686658 24820 raft_consensus.cc:2454] T e3fd481f3c78497eb629b0180c07674f P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:00.686942 25150 leader_election.cc:302] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:00.687134 25429 raft_consensus.cc:2781] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:00.687222 25429 raft_consensus.cc:686] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:00.687369 25429 consensus_queue.cc:237] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.687909 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3fd481f3c78497eb629b0180c07674f" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:00.687973 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3fd481f3c78497eb629b0180c07674f" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:00.688021 25083 raft_consensus.cc:2454] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:00.688021 22228 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:00.689771 22227 catalog_manager.cc:5484] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:00.694478 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac6c4b48e8b4c5fb0866222e8df6e39 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d17b69c6d9f54b1abe913a18b35cbbe5]), partition=
I20240502 07:59:00.694782 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac6c4b48e8b4c5fb0866222e8df6e39. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.694806 25195 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac6c4b48e8b4c5fb0866222e8df6e39 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d17b69c6d9f54b1abe913a18b35cbbe5]), partition=
I20240502 07:59:00.695189 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac6c4b48e8b4c5fb0866222e8df6e39. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.696733 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 0ac6c4b48e8b4c5fb0866222e8df6e39 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d17b69c6d9f54b1abe913a18b35cbbe5]), partition=
I20240502 07:59:00.697054 26626 tablet_bootstrap.cc:492] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:00.697099 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ac6c4b48e8b4c5fb0866222e8df6e39. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:00.698058 26626 tablet_bootstrap.cc:654] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.699286 26626 tablet_bootstrap.cc:492] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:00.699376 26626 ts_tablet_manager.cc:1367] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:00.699836 26626 raft_consensus.cc:348] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:00.699942 26626 raft_consensus.cc:374] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.699971 26626 raft_consensus.cc:729] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.700085 26626 consensus_queue.cc:260] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:00.700157 26737 tablet_bootstrap.cc:492] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:00.700393 26626 ts_tablet_manager.cc:1398] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.700949 26737 tablet_bootstrap.cc:654] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.701773 26413 tablet_bootstrap.cc:492] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:00.702102 26737 tablet_bootstrap.cc:492] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:00.702186 26737 ts_tablet_manager.cc:1367] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:00.702555 26413 tablet_bootstrap.cc:654] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:00.702598 26737 raft_consensus.cc:348] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:00.702708 26737 raft_consensus.cc:374] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.702751 26737 raft_consensus.cc:729] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.702869 26737 consensus_queue.cc:260] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:00.703101 26737 ts_tablet_manager.cc:1398] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:00.703591 26413 tablet_bootstrap.cc:492] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:00.703687 26413 ts_tablet_manager.cc:1367] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:00.704175 22227 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_18" } modify_external_catalogs: true
I20240502 07:59:00.704289 22227 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalance_test_table_18" } modify_external_catalogs: true
I20240502 07:59:00.704351 26413 raft_consensus.cc:348] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:00.704448 26413 raft_consensus.cc:374] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:00.704478 26413 raft_consensus.cc:729] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:00.704591 26413 consensus_queue.cc:260] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:00.704870 26413 ts_tablet_manager.cc:1398] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:00.707876 22227 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 4 replicas of tablet 4dc697950c8140878b50550fe0ff458d
I20240502 07:59:00.708222 22227 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet e5c1730dba9645ac90c5bb10def836a2
I20240502 07:59:00.708446 22227 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 132ac92527d24e18b7e4554cf47bbc01
I20240502 07:59:00.708868 24800 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dc697950c8140878b50550fe0ff458d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:59:00.708958 22744 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dc697950c8140878b50550fe0ff458d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:00.709017 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet e5c1730dba9645ac90c5bb10def836a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:00.709054 22743 tablet_service.cc:1514] Processing DeleteTablet for tablet 132ac92527d24e18b7e4554cf47bbc01 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:00.709069 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet e5c1730dba9645ac90c5bb10def836a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:00.709208 26935 tablet_replica.cc:330] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:59:00.709223 26659 tablet_replica.cc:330] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:00.709231 25812 tablet_replica.cc:330] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:00.709370 26659 raft_consensus.cc:2229] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:00.709372 26935 raft_consensus.cc:2229] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.709370 25812 raft_consensus.cc:2229] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.709491 26935 raft_consensus.cc:2258] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.709527 25812 raft_consensus.cc:2258] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.709570 26659 raft_consensus.cc:2258] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.710027 26659 ts_tablet_manager.cc:1875] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.710024 25812 ts_tablet_manager.cc:1875] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.710129 26935 ts_tablet_manager.cc:1875] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.712225 22350 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dc697950c8140878b50550fe0ff458d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:00.712226 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet e5c1730dba9645ac90c5bb10def836a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:00.712226 22352 tablet_service.cc:1514] Processing DeleteTablet for tablet 132ac92527d24e18b7e4554cf47bbc01 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:00.712450 26075 tablet_replica.cc:330] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:00.712584 26075 raft_consensus.cc:2229] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:00.712664 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dc697950c8140878b50550fe0ff458d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:00.712721 22482 tablet_service.cc:1514] Processing DeleteTablet for tablet 132ac92527d24e18b7e4554cf47bbc01 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:00 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:00.712828 26811 tablet_replica.cc:330] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:00.712844 26075 raft_consensus.cc:2258] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.712935 26811 raft_consensus.cc:2229] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.713068 26811 raft_consensus.cc:2258] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.713407 26075 ts_tablet_manager.cc:1875] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.713547 26935 ts_tablet_manager.cc:1888] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:00.713609 26935 log.cc:1196] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/4dc697950c8140878b50550fe0ff458d
I20240502 07:59:00.713616 26811 ts_tablet_manager.cc:1875] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.713902 26935 ts_tablet_manager.cc:1909] T 4dc697950c8140878b50550fe0ff458d P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:59:00.714865 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet 4dc697950c8140878b50550fe0ff458d (table rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]) successfully deleted
I20240502 07:59:00.715883 26075 ts_tablet_manager.cc:1888] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:00.715893 26811 ts_tablet_manager.cc:1888] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:00.715958 26811 log.cc:1196] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/4dc697950c8140878b50550fe0ff458d
I20240502 07:59:00.715958 26075 log.cc:1196] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/4dc697950c8140878b50550fe0ff458d
I20240502 07:59:00.716214 26075 ts_tablet_manager.cc:1909] T 4dc697950c8140878b50550fe0ff458d P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:59:00.716220 26811 ts_tablet_manager.cc:1909] T 4dc697950c8140878b50550fe0ff458d P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:59:00.717204 25812 ts_tablet_manager.cc:1888] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:00.717504 26811 tablet_replica.cc:330] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:00.717540 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 4dc697950c8140878b50550fe0ff458d (table rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]) successfully deleted
I20240502 07:59:00.717630 26811 raft_consensus.cc:2229] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.717751 26811 raft_consensus.cc:2258] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.717787 26659 ts_tablet_manager.cc:1888] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:59:00.718225 26811 ts_tablet_manager.cc:1875] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.720208 26811 ts_tablet_manager.cc:1888] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:59:00.720276 26811 log.cc:1196] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/132ac92527d24e18b7e4554cf47bbc01
I20240502 07:59:00.720556 26811 ts_tablet_manager.cc:1909] T 132ac92527d24e18b7e4554cf47bbc01 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:59:00.721231 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 132ac92527d24e18b7e4554cf47bbc01 (table rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]) successfully deleted
I20240502 07:59:00.717862 26659 log.cc:1196] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/e5c1730dba9645ac90c5bb10def836a2
I20240502 07:59:00.721709 25812 log.cc:1196] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/4dc697950c8140878b50550fe0ff458d
I20240502 07:59:00.722011 26659 ts_tablet_manager.cc:1909] T e5c1730dba9645ac90c5bb10def836a2 P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:59:00.722011 25812 ts_tablet_manager.cc:1909] T 4dc697950c8140878b50550fe0ff458d P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:59:00.722658 25812 tablet_replica.cc:330] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:00.722735 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 4dc697950c8140878b50550fe0ff458d (table rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]) successfully deleted
I20240502 07:59:00.722846 25812 raft_consensus.cc:2229] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.723062 25812 raft_consensus.cc:2258] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.723541 25812 ts_tablet_manager.cc:1875] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.724156 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet e5c1730dba9645ac90c5bb10def836a2 (table rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]) successfully deleted
I20240502 07:59:00.725807 26075 tablet_replica.cc:330] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:00.726015 26075 raft_consensus.cc:2229] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.726044 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 4dc697950c8140878b50550fe0ff458d (table rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]) successfully deleted
I20240502 07:59:00.726207 26075 raft_consensus.cc:2258] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.726768 26075 ts_tablet_manager.cc:1875] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.727437 25064 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:00.728778 26075 ts_tablet_manager.cc:1888] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:59:00.728847 26075 log.cc:1196] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/e5c1730dba9645ac90c5bb10def836a2
I20240502 07:59:00.729087 24932 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:00.729218 26075 ts_tablet_manager.cc:1909] T e5c1730dba9645ac90c5bb10def836a2 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:59:00.730775 25812 ts_tablet_manager.cc:1888] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:59:00.730895 25812 log.cc:1196] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/e5c1730dba9645ac90c5bb10def836a2
I20240502 07:59:00.731168 25812 ts_tablet_manager.cc:1909] T e5c1730dba9645ac90c5bb10def836a2 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:59:00.731712 25812 tablet_replica.cc:330] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:00.731827 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet e5c1730dba9645ac90c5bb10def836a2 (table rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]) successfully deleted
I20240502 07:59:00.731875 25812 raft_consensus.cc:2229] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:00.732106 25812 raft_consensus.cc:2258] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.732563 25812 ts_tablet_manager.cc:1875] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.734548 25812 ts_tablet_manager.cc:1888] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:59:00.734663 25812 log.cc:1196] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/132ac92527d24e18b7e4554cf47bbc01
I20240502 07:59:00.734943 25812 ts_tablet_manager.cc:1909] T 132ac92527d24e18b7e4554cf47bbc01 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:59:00.735538 26075 tablet_replica.cc:330] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:00.735710 26075 raft_consensus.cc:2229] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.735862 26075 raft_consensus.cc:2258] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.736318 26075 ts_tablet_manager.cc:1875] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:00.738582 26075 ts_tablet_manager.cc:1888] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20240502 07:59:00.738708 26075 log.cc:1196] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/132ac92527d24e18b7e4554cf47bbc01
I20240502 07:59:00.738989 26075 ts_tablet_manager.cc:1909] T 132ac92527d24e18b7e4554cf47bbc01 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:59:00.739953 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet e5c1730dba9645ac90c5bb10def836a2 (table rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]) successfully deleted
I20240502 07:59:00.740118 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 132ac92527d24e18b7e4554cf47bbc01 (table rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]) successfully deleted
I20240502 07:59:00.742007 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 132ac92527d24e18b7e4554cf47bbc01 (table rebalance_test_table_18 [id=4ae19d06ad284921b8f132a6449508a4]) successfully deleted
I20240502 07:59:00.743717 25195 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:00.762516 22352 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:00.772672 22482 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:00.772995 24800 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:00.774520 22743 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:00.794845 22612 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:00.835785 22634 consensus_queue.cc:237] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:00.837096 25084 raft_consensus.cc:1261] T d9c0c75053be41f59786f1cf4fb3062c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:00.837168 22372 raft_consensus.cc:1261] T d9c0c75053be41f59786f1cf4fb3062c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:00.837644 25587 consensus_queue.cc:1035] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:00.837807 26771 consensus_queue.cc:1035] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:00.839540 25587 raft_consensus.cc:2932] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:00.839879 22372 raft_consensus.cc:2932] T d9c0c75053be41f59786f1cf4fb3062c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:00.840973 25083 raft_consensus.cc:2932] T d9c0c75053be41f59786f1cf4fb3062c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:00.842120 22224 catalog_manager.cc:5484] T d9c0c75053be41f59786f1cf4fb3062c P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index 4 to 5, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:00.846345 22743 tablet_service.cc:1514] Processing DeleteTablet for tablet d9c0c75053be41f59786f1cf4fb3062c with delete_type TABLET_DATA_TOMBSTONED (TS f00f3a1876444ad2bf9c919068ab8014 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:00.846647 25812 tablet_replica.cc:330] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:00.846868 25812 raft_consensus.cc:2229] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:00.847033 25812 raft_consensus.cc:2258] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:00.847355 26192 raft_consensus.cc:482] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:00.847492 26192 raft_consensus.cc:504] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:00.847621 25812 ts_tablet_manager.cc:1875] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:00.847930 26192 leader_election.cc:288] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:00.848430 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac6c4b48e8b4c5fb0866222e8df6e39" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:00.848588 24952 raft_consensus.cc:2454] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:00.848922 24757 leader_election.cc:302] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:00.849169 26192 raft_consensus.cc:2781] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:00.849224 26192 raft_consensus.cc:482] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:00.849272 26192 raft_consensus.cc:3037] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.850075 25812 ts_tablet_manager.cc:1888] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:00.850140 25812 log.cc:1196] T d9c0c75053be41f59786f1cf4fb3062c P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/d9c0c75053be41f59786f1cf4fb3062c
I20240502 07:59:00.850188 26192 raft_consensus.cc:504] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:00.850564 26192 leader_election.cc:288] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:00.850831 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet d9c0c75053be41f59786f1cf4fb3062c (table rebalance_test_table_3 [id=2c887f37ddf349f3bbec2b1330f05fc7]) successfully deleted
I20240502 07:59:00.851116 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac6c4b48e8b4c5fb0866222e8df6e39" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:00.851241 24952 raft_consensus.cc:3037] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.852242 24952 raft_consensus.cc:2454] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:00.852655 24757 leader_election.cc:302] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:00.853003 26192 raft_consensus.cc:2781] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:00.852953 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac6c4b48e8b4c5fb0866222e8df6e39" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:00.853093 25215 raft_consensus.cc:3037] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:00.853144 26192 raft_consensus.cc:686] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:59:00.853287 26192 consensus_queue.cc:237] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:00.852953 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ac6c4b48e8b4c5fb0866222e8df6e39" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:00.854065 25215 raft_consensus.cc:2454] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:00.855333 22225 catalog_manager.cc:5484] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:00.933696 24932 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:00.937093 22352 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:00.937189 25064 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:00.949560 24800 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:00.952164 25195 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:00.965922 22612 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:00.966563 22743 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:00.973647 22482 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:00.976575 22227 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:00.981058 25214 raft_consensus.cc:1261] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 4b4e7eeb7f3f42119c0363a79febc54b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:00.981595 27007 consensus_queue.cc:1035] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:00.985100 22227 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 0ac6c4b48e8b4c5fb0866222e8df6e39 (table rebalancer_dynamic_table [id=d17b69c6d9f54b1abe913a18b35cbbe5]). Expected version 1 got 0
I20240502 07:59:00.989763 27057 tablet.cc:1722] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:00.990202 27057 tablet.cc:1620] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:00.990912 22227 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 0ac6c4b48e8b4c5fb0866222e8df6e39 (table rebalancer_dynamic_table [id=d17b69c6d9f54b1abe913a18b35cbbe5]). Expected version 1 got 0
I20240502 07:59:00.996860 22227 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=d17b69c6d9f54b1abe913a18b35cbbe5] alter complete (version 1)
I20240502 07:59:00.997556 27056 tablet.cc:1722] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:00.997916 27056 tablet.cc:1620] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:00.998111 24952 raft_consensus.cc:1261] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer 4b4e7eeb7f3f42119c0363a79febc54b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:00.998602 27058 consensus_queue.cc:1035] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:01.001760 27061 tablet.cc:1722] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:01.002089 27061 tablet.cc:1620] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:01.002236 22227 catalog_manager.cc:5576] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995) does not have the latest schema for tablet 0ac6c4b48e8b4c5fb0866222e8df6e39 (table rebalancer_dynamic_table [id=d17b69c6d9f54b1abe913a18b35cbbe5]). Expected version 1 got 0
I20240502 07:59:01.029855 22764 consensus_queue.cc:237] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:01.031128 24952 raft_consensus.cc:1261] T 4e202e9cf0154eb9814a7e3ad93f1e42 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:01.031399 22503 raft_consensus.cc:1261] T 4e202e9cf0154eb9814a7e3ad93f1e42 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:01.031512 26602 consensus_queue.cc:1035] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:01.031762 24452 consensus_queue.cc:1035] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:01.033414 26602 raft_consensus.cc:2932] T 4e202e9cf0154eb9814a7e3ad93f1e42 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:01.033803 24952 raft_consensus.cc:2932] T 4e202e9cf0154eb9814a7e3ad93f1e42 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:01.034027 22503 raft_consensus.cc:2932] T 4e202e9cf0154eb9814a7e3ad93f1e42 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:01.036422 22223 catalog_manager.cc:5484] T 4e202e9cf0154eb9814a7e3ad93f1e42 P 1aecef81733c47b3b468504934357231 reported cstate change: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:01.039124 22612 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e202e9cf0154eb9814a7e3ad93f1e42 with delete_type TABLET_DATA_TOMBSTONED (TS e51b10af49484aaa8f0267aebc1bf27f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:01.039346 26659 tablet_replica.cc:330] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:01.039479 26659 raft_consensus.cc:2229] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:01.039614 26659 raft_consensus.cc:2258] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:01.040148 26659 ts_tablet_manager.cc:1875] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:01.042309 26659 ts_tablet_manager.cc:1888] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:01.042393 26659 log.cc:1196] T 4e202e9cf0154eb9814a7e3ad93f1e42 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/4e202e9cf0154eb9814a7e3ad93f1e42
I20240502 07:59:01.042939 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 4e202e9cf0154eb9814a7e3ad93f1e42 (table rebalance_test_table_5 [id=80b9c0f932754699a4f41d2c2188b70b]) successfully deleted
I20240502 07:59:01.076745 25429 consensus_queue.cc:1035] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:01.080080 25083 raft_consensus.cc:3037] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:01.086619 26720 consensus_queue.cc:1035] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:01.088474 26758 consensus_queue.cc:1035] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:01.101174 22230 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:01.101274 26758 consensus_queue.cc:1035] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:01.108496 27056 tablet.cc:1722] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:01.108551 27057 tablet.cc:1722] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:01.108840 27056 tablet.cc:1620] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:01.108850 27057 tablet.cc:1620] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:01.110424 27061 tablet.cc:1722] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:01.110764 27061 tablet.cc:1620] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:01.116545 26497 consensus_queue.cc:1035] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:01.119056 22228 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=d17b69c6d9f54b1abe913a18b35cbbe5] alter complete (version 2)
I20240502 07:59:01.120983 26497 consensus_queue.cc:1035] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:01.141503 25064 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:01.144243 25195 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:01.152709 22482 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:01.155293 22612 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:01.158851 22743 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:01.159137 24932 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:01.159670 22352 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:01.167097 24800 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:01.222656 22228 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:01.224628 22223 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:01.227560 22372 consensus_queue.cc:237] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:01.228890 24952 raft_consensus.cc:1261] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:01.229348 22634 raft_consensus.cc:1261] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:01.229557 26558 consensus_queue.cc:1035] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:01.229844 27056 tablet.cc:1722] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:01.230628 26558 consensus_queue.cc:1035] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:01.232878 27057 tablet.cc:1722] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:01.234212 27061 tablet.cc:1722] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:01.240048 22228 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=d17b69c6d9f54b1abe913a18b35cbbe5] alter complete (version 3)
I20240502 07:59:01.241592 25195 tablet_service.cc:1467] Processing CreateTablet for tablet a64141e51eb14b04a19cedfa5f346e79 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=97589e13ee074648b68057aa77a8d10d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:01.241871 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a64141e51eb14b04a19cedfa5f346e79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:01.241271 26322 raft_consensus.cc:2932] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:01.244134 24951 raft_consensus.cc:2932] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:01.245018 22634 raft_consensus.cc:2932] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:01.245299 22228 catalog_manager.cc:5484] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:01.246167 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 751f8c9cbf684b01b90e7bd26685f8bf (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=97589e13ee074648b68057aa77a8d10d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:01.246435 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 751f8c9cbf684b01b90e7bd26685f8bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:01.248692 27159 tablet_bootstrap.cc:492] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:01.250527 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 5242aa2d2d9c477994026edb0d42051c (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=97589e13ee074648b68057aa77a8d10d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:01.250527 22352 tablet_service.cc:1467] Processing CreateTablet for tablet a64141e51eb14b04a19cedfa5f346e79 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=97589e13ee074648b68057aa77a8d10d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:01.250799 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a64141e51eb14b04a19cedfa5f346e79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:01.250804 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5242aa2d2d9c477994026edb0d42051c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:01.253535 22482 tablet_service.cc:1467] Processing CreateTablet for tablet a64141e51eb14b04a19cedfa5f346e79 (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=97589e13ee074648b68057aa77a8d10d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:01.253822 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a64141e51eb14b04a19cedfa5f346e79. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:01.253881 25064 tablet_service.cc:1467] Processing CreateTablet for tablet 751f8c9cbf684b01b90e7bd26685f8bf (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=97589e13ee074648b68057aa77a8d10d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:01.254042 27166 tablet_bootstrap.cc:492] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:01.254119 25064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 751f8c9cbf684b01b90e7bd26685f8bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:01.253880 27159 tablet_bootstrap.cc:654] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:01.254837 27166 tablet_bootstrap.cc:654] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:01.255199 27165 tablet_bootstrap.cc:492] T a64141e51eb14b04a19cedfa5f346e79 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:01.256031 27169 tablet_bootstrap.cc:492] T a64141e51eb14b04a19cedfa5f346e79 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:01.256392 27165 tablet_bootstrap.cc:654] T a64141e51eb14b04a19cedfa5f346e79 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:01.256803 27169 tablet_bootstrap.cc:654] T a64141e51eb14b04a19cedfa5f346e79 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:01.257357 27170 tablet_bootstrap.cc:492] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:01.257476 25195 tablet_service.cc:1467] Processing CreateTablet for tablet 5242aa2d2d9c477994026edb0d42051c (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=97589e13ee074648b68057aa77a8d10d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:01.257743 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5242aa2d2d9c477994026edb0d42051c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:01.257771 27159 tablet_bootstrap.cc:492] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:01.257849 27159 ts_tablet_manager.cc:1367] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 07:59:01.258344 27170 tablet_bootstrap.cc:654] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:01.258293 27159 raft_consensus.cc:348] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:01.258404 27159 raft_consensus.cc:374] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:01.258433 27159 raft_consensus.cc:729] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:01.258610 27159 consensus_queue.cc:260] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:01.259442 27159 ts_tablet_manager.cc:1398] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:01.259560 27169 tablet_bootstrap.cc:492] T a64141e51eb14b04a19cedfa5f346e79 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:01.259680 27169 ts_tablet_manager.cc:1367] T a64141e51eb14b04a19cedfa5f346e79 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:01.260293 27165 tablet_bootstrap.cc:492] T a64141e51eb14b04a19cedfa5f346e79 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:01.260416 27165 ts_tablet_manager.cc:1367] T a64141e51eb14b04a19cedfa5f346e79 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:01.260107 27169 raft_consensus.cc:348] T a64141e51eb14b04a19cedfa5f346e79 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:01.260560 27169 raft_consensus.cc:374] T a64141e51eb14b04a19cedfa5f346e79 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:01.260644 27169 raft_consensus.cc:729] T a64141e51eb14b04a19cedfa5f346e79 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:01.260799 27169 consensus_queue.cc:260] T a64141e51eb14b04a19cedfa5f346e79 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:01.260859 27165 raft_consensus.cc:348] T a64141e51eb14b04a19cedfa5f346e79 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:01.260962 27165 raft_consensus.cc:374] T a64141e51eb14b04a19cedfa5f346e79 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:01.261044 27165 raft_consensus.cc:729] T a64141e51eb14b04a19cedfa5f346e79 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:01.261216 27165 consensus_queue.cc:260] T a64141e51eb14b04a19cedfa5f346e79 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:01.261735 27169 ts_tablet_manager.cc:1398] T a64141e51eb14b04a19cedfa5f346e79 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:01.263118 27166 tablet_bootstrap.cc:492] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:01.263116 27165 ts_tablet_manager.cc:1398] T a64141e51eb14b04a19cedfa5f346e79 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:01.263214 27166 ts_tablet_manager.cc:1367] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 07:59:01.263640 27166 raft_consensus.cc:348] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:01.263721 27166 raft_consensus.cc:374] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:01.263741 27166 raft_consensus.cc:729] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:01.263847 27166 consensus_queue.cc:260] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:01.264098 27166 ts_tablet_manager.cc:1398] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:01.265003 27170 tablet_bootstrap.cc:492] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:01.265136 27170 ts_tablet_manager.cc:1367] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:59:01.265889 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 751f8c9cbf684b01b90e7bd26685f8bf (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=97589e13ee074648b68057aa77a8d10d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:01.266177 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 751f8c9cbf684b01b90e7bd26685f8bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:01.266296 27170 raft_consensus.cc:348] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:01.266405 27170 raft_consensus.cc:374] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:01.266440 27170 raft_consensus.cc:729] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:01.266573 27170 consensus_queue.cc:260] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:01.267365 27170 ts_tablet_manager.cc:1398] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:01.267590 24931 tablet_service.cc:1467] Processing CreateTablet for tablet 5242aa2d2d9c477994026edb0d42051c (DEFAULT_TABLE table=rebalancer_extra_table_12 [id=97589e13ee074648b68057aa77a8d10d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:01.267792 24931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5242aa2d2d9c477994026edb0d42051c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:01.272606 22482 tablet_service.cc:1514] Processing DeleteTablet for tablet 45925e9713cb4b748efde06f77cf3702 with delete_type TABLET_DATA_TOMBSTONED (TS fabddb16c85944648a710ec52fc7244c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:01.273026 27176 tablet_replica.cc:330] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:01.273571 27176 raft_consensus.cc:2229] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:01.273748 27158 tablet_bootstrap.cc:492] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:01.273809 27176 raft_consensus.cc:2258] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:01.274618 27176 ts_tablet_manager.cc:1875] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:01.275013 27158 tablet_bootstrap.cc:654] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:01.276230 27175 tablet_bootstrap.cc:492] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:01.277437 27175 tablet_bootstrap.cc:654] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:01.277532 27176 ts_tablet_manager.cc:1888] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:01.277647 27176 log.cc:1196] T 45925e9713cb4b748efde06f77cf3702 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/45925e9713cb4b748efde06f77cf3702
I20240502 07:59:01.277930 27158 tablet_bootstrap.cc:492] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:01.278093 27158 ts_tablet_manager.cc:1367] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:01.278640 27158 raft_consensus.cc:348] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:01.278806 27158 raft_consensus.cc:374] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:01.278873 27158 raft_consensus.cc:729] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:01.279050 27158 consensus_queue.cc:260] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:01.279400 27158 ts_tablet_manager.cc:1398] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:01.279495 27175 tablet_bootstrap.cc:492] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:01.279623 27158 tablet_bootstrap.cc:492] T 5242aa2d2d9c477994026edb0d42051c P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:01.279618 27175 ts_tablet_manager.cc:1367] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 07:59:01.280185 27175 raft_consensus.cc:348] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:01.280320 27175 raft_consensus.cc:374] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:01.280447 27175 raft_consensus.cc:729] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:01.280623 27175 consensus_queue.cc:260] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:01.280705 27158 tablet_bootstrap.cc:654] T 5242aa2d2d9c477994026edb0d42051c P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:01.281086 27175 ts_tablet_manager.cc:1398] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:01.281351 27175 tablet_bootstrap.cc:492] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:01.281442 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 45925e9713cb4b748efde06f77cf3702 (table rebalance_test_table_7 [id=255d8aedf8f845a1b8080105bc5b3a93]) successfully deleted
I20240502 07:59:01.282642 27175 tablet_bootstrap.cc:654] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:01.283327 27158 tablet_bootstrap.cc:492] T 5242aa2d2d9c477994026edb0d42051c P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:01.283418 27158 ts_tablet_manager.cc:1367] T 5242aa2d2d9c477994026edb0d42051c P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20240502 07:59:01.283814 27158 raft_consensus.cc:348] T 5242aa2d2d9c477994026edb0d42051c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:01.283923 27158 raft_consensus.cc:374] T 5242aa2d2d9c477994026edb0d42051c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:01.283949 27158 raft_consensus.cc:729] T 5242aa2d2d9c477994026edb0d42051c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:01.284054 27158 consensus_queue.cc:260] T 5242aa2d2d9c477994026edb0d42051c P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:01.284300 27158 ts_tablet_manager.cc:1398] T 5242aa2d2d9c477994026edb0d42051c P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:01.286707 27175 tablet_bootstrap.cc:492] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:01.287381 27175 ts_tablet_manager.cc:1367] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:01.287988 27175 raft_consensus.cc:348] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:01.288101 27175 raft_consensus.cc:374] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:01.288131 27175 raft_consensus.cc:729] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:01.288240 27175 consensus_queue.cc:260] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:01.288496 27175 ts_tablet_manager.cc:1398] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:01.324493 25429 raft_consensus.cc:482] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:01.324626 25429 raft_consensus.cc:504] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:01.325109 25429 leader_election.cc:288] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:01.326167 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a64141e51eb14b04a19cedfa5f346e79" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:01.326323 22372 raft_consensus.cc:2454] T a64141e51eb14b04a19cedfa5f346e79 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:01.326637 25151 leader_election.cc:302] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:01.327014 25429 raft_consensus.cc:2781] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:01.327081 25429 raft_consensus.cc:482] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:01.327122 25429 raft_consensus.cc:3037] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:01.328226 25429 raft_consensus.cc:504] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:01.328612 25429 leader_election.cc:288] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:01.329078 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a64141e51eb14b04a19cedfa5f346e79" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:01.329193 22372 raft_consensus.cc:3037] T a64141e51eb14b04a19cedfa5f346e79 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:01.330230 22372 raft_consensus.cc:2454] T a64141e51eb14b04a19cedfa5f346e79 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:01.330591 25151 leader_election.cc:302] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:01.330754 25429 raft_consensus.cc:2781] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:01.330834 25429 raft_consensus.cc:686] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:01.330958 25429 consensus_queue.cc:237] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:01.333266 22222 catalog_manager.cc:5484] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:01.340704 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a64141e51eb14b04a19cedfa5f346e79" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:01.340869 22503 raft_consensus.cc:2454] T a64141e51eb14b04a19cedfa5f346e79 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:01.341089 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a64141e51eb14b04a19cedfa5f346e79" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:01.341194 22502 raft_consensus.cc:3037] T a64141e51eb14b04a19cedfa5f346e79 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:01.342329 22502 raft_consensus.cc:2454] T a64141e51eb14b04a19cedfa5f346e79 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:01.365921 22229 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:01.366066 22229 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:01.368033 22229 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 0ac6c4b48e8b4c5fb0866222e8df6e39
I20240502 07:59:01.368682 24800 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac6c4b48e8b4c5fb0866222e8df6e39 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:01 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:59:01.368763 24931 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac6c4b48e8b4c5fb0866222e8df6e39 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:01 UTC) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:01.368839 25195 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ac6c4b48e8b4c5fb0866222e8df6e39 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:01 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:01.369160 27211 tablet_replica.cc:330] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:59:01.369309 27211 raft_consensus.cc:2229] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:01.369408 27213 tablet_replica.cc:330] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:01.369570 27211 raft_consensus.cc:2258] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:01.369568 27213 raft_consensus.cc:2229] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:01.369623 27212 tablet_replica.cc:330] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:01.369797 27213 raft_consensus.cc:2258] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:01.369813 27212 raft_consensus.cc:2229] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:01.370002 27212 raft_consensus.cc:2258] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:01.370299 27211 ts_tablet_manager.cc:1875] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:01.370347 27213 ts_tablet_manager.cc:1875] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:01.371203 27212 ts_tablet_manager.cc:1875] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:01.372764 27213 ts_tablet_manager.cc:1888] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:01.372845 27213 log.cc:1196] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/0ac6c4b48e8b4c5fb0866222e8df6e39
I20240502 07:59:01.372859 27211 ts_tablet_manager.cc:1888] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:01.372970 27211 log.cc:1196] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/0ac6c4b48e8b4c5fb0866222e8df6e39
I20240502 07:59:01.373205 27213 ts_tablet_manager.cc:1909] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:59:01.373309 27211 ts_tablet_manager.cc:1909] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:59:01.373560 27212 ts_tablet_manager.cc:1888] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:01.373636 27212 log.cc:1196] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/0ac6c4b48e8b4c5fb0866222e8df6e39
I20240502 07:59:01.373978 27212 ts_tablet_manager.cc:1909] T 0ac6c4b48e8b4c5fb0866222e8df6e39 P 1aecef81733c47b3b468504934357231: Deleting consensus metadata
I20240502 07:59:01.373948 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 0ac6c4b48e8b4c5fb0866222e8df6e39 (table rebalancer_dynamic_table_new_name [id=d17b69c6d9f54b1abe913a18b35cbbe5]) successfully deleted
I20240502 07:59:01.374085 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet 0ac6c4b48e8b4c5fb0866222e8df6e39 (table rebalancer_dynamic_table_new_name [id=d17b69c6d9f54b1abe913a18b35cbbe5]) successfully deleted
I20240502 07:59:01.374706 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet 0ac6c4b48e8b4c5fb0866222e8df6e39 (table rebalancer_dynamic_table_new_name [id=d17b69c6d9f54b1abe913a18b35cbbe5]) successfully deleted
I20240502 07:59:01.430866 26192 raft_consensus.cc:482] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:01.430989 26192 raft_consensus.cc:504] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:01.431360 26192 leader_election.cc:288] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:01.431717 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751f8c9cbf684b01b90e7bd26685f8bf" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:01.431785 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751f8c9cbf684b01b90e7bd26685f8bf" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:01.431851 25083 raft_consensus.cc:2454] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:01.431910 24951 raft_consensus.cc:2454] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:01.432124 24754 leader_election.cc:302] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:01.432296 26192 raft_consensus.cc:2781] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:01.432343 26192 raft_consensus.cc:482] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:01.432394 26192 raft_consensus.cc:3037] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:01.433233 26192 raft_consensus.cc:504] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:01.433593 26192 leader_election.cc:288] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:01.433888 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751f8c9cbf684b01b90e7bd26685f8bf" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:01.433918 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751f8c9cbf684b01b90e7bd26685f8bf" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:01.434005 24951 raft_consensus.cc:3037] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:01.434051 25083 raft_consensus.cc:3037] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:01.434852 24951 raft_consensus.cc:2454] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:01.434878 25083 raft_consensus.cc:2454] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:01.435178 24757 leader_election.cc:302] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:01.435375 26192 raft_consensus.cc:2781] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:01.435472 26192 raft_consensus.cc:686] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:59:01.435638 26192 consensus_queue.cc:237] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:01.437417 22229 catalog_manager.cc:5484] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:01.469890 22229 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:01.474938 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 45a9d5b38fdb44839162b938763e3ad6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a4d3a04c126c45e58f86616c2c8010b2]), partition=
I20240502 07:59:01.475065 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 45a9d5b38fdb44839162b938763e3ad6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a4d3a04c126c45e58f86616c2c8010b2]), partition=
I20240502 07:59:01.475054 25064 tablet_service.cc:1467] Processing CreateTablet for tablet 45a9d5b38fdb44839162b938763e3ad6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a4d3a04c126c45e58f86616c2c8010b2]), partition=
I20240502 07:59:01.475261 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45a9d5b38fdb44839162b938763e3ad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:01.475281 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45a9d5b38fdb44839162b938763e3ad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:01.475291 25064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45a9d5b38fdb44839162b938763e3ad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:01.476902 27170 tablet_bootstrap.cc:492] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:01.477351 27165 tablet_bootstrap.cc:492] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:01.477350 27159 tablet_bootstrap.cc:492] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:01.477811 27170 tablet_bootstrap.cc:654] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:01.478096 27159 tablet_bootstrap.cc:654] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:01.478096 27165 tablet_bootstrap.cc:654] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:01.479135 27165 tablet_bootstrap.cc:492] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:01.479135 27159 tablet_bootstrap.cc:492] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:01.479220 27159 ts_tablet_manager.cc:1367] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:01.479220 27165 ts_tablet_manager.cc:1367] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:01.479679 27165 raft_consensus.cc:348] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:01.479655 27159 raft_consensus.cc:348] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:01.479774 27165 raft_consensus.cc:374] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:01.479774 27159 raft_consensus.cc:374] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:01.479808 27165 raft_consensus.cc:729] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:01.479808 27159 raft_consensus.cc:729] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:01.479926 27159 consensus_queue.cc:260] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:01.479926 27165 consensus_queue.cc:260] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:01.480193 27165 ts_tablet_manager.cc:1398] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:01.480192 27159 ts_tablet_manager.cc:1398] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:01.481066 27170 tablet_bootstrap.cc:492] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:01.481166 27170 ts_tablet_manager.cc:1367] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20240502 07:59:01.481731 27170 raft_consensus.cc:348] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:01.481832 27170 raft_consensus.cc:374] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:01.481870 27170 raft_consensus.cc:729] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:01.481995 27170 consensus_queue.cc:260] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:01.482333 27170 ts_tablet_manager.cc:1398] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:01.560631 27157 raft_consensus.cc:482] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:01.560786 27157 raft_consensus.cc:504] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:01.561255 27157 leader_election.cc:288] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:01.561838 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45a9d5b38fdb44839162b938763e3ad6" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:01.561985 24820 raft_consensus.cc:2454] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:01.562494 22307 leader_election.cc:302] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:01.562629 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45a9d5b38fdb44839162b938763e3ad6" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:01.562767 25083 raft_consensus.cc:2454] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:01.562786 27157 raft_consensus.cc:2781] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:01.562839 27157 raft_consensus.cc:482] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:01.562875 27157 raft_consensus.cc:3037] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:01.564029 27157 raft_consensus.cc:504] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:01.564349 27157 leader_election.cc:288] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:01.564658 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45a9d5b38fdb44839162b938763e3ad6" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:01.564687 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45a9d5b38fdb44839162b938763e3ad6" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:01.564762 25083 raft_consensus.cc:3037] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:01.564803 24820 raft_consensus.cc:3037] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:01.565549 25083 raft_consensus.cc:2454] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:01.565583 24820 raft_consensus.cc:2454] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:01.565830 22305 leader_election.cc:302] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:01.565982 27157 raft_consensus.cc:2781] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:01.566067 27157 raft_consensus.cc:686] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:59:01.566236 27157 consensus_queue.cc:237] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:01.568097 22227 catalog_manager.cc:5484] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:01.586719 27218 raft_consensus.cc:482] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:01.586838 27218 raft_consensus.cc:504] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:01.587217 27218 leader_election.cc:288] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:01.587642 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5242aa2d2d9c477994026edb0d42051c" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:01.587666 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5242aa2d2d9c477994026edb0d42051c" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:01.587785 24951 raft_consensus.cc:2454] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:01.587785 25214 raft_consensus.cc:2454] T 5242aa2d2d9c477994026edb0d42051c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:01.588088 22701 leader_election.cc:302] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:01.588239 27218 raft_consensus.cc:2781] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:01.588294 27218 raft_consensus.cc:482] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:01.588362 27218 raft_consensus.cc:3037] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:01.589370 27218 raft_consensus.cc:504] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:01.589743 27218 leader_election.cc:288] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:01.590067 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5242aa2d2d9c477994026edb0d42051c" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:01.590122 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5242aa2d2d9c477994026edb0d42051c" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:01.590189 25214 raft_consensus.cc:3037] T 5242aa2d2d9c477994026edb0d42051c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:01.590230 24951 raft_consensus.cc:3037] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:01.591171 24951 raft_consensus.cc:2454] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:01.591171 25214 raft_consensus.cc:2454] T 5242aa2d2d9c477994026edb0d42051c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:01.591473 22701 leader_election.cc:302] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:01.591652 27218 raft_consensus.cc:2781] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:01.591738 27218 raft_consensus.cc:686] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:59:01.591888 27218 consensus_queue.cc:237] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:01.593660 22227 catalog_manager.cc:5484] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:01.620874 24931 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:01.626428 25195 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:01.628589 25064 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:01.635361 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:01.640444 24800 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:01.640600 22352 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:01.641259 22612 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:01.660189 22482 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:01.679237 22227 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:01.684878 25083 raft_consensus.cc:1261] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:01.685235 24820 raft_consensus.cc:1261] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:01.685300 27157 consensus_queue.cc:1035] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:01.686291 27157 consensus_queue.cc:1035] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:01.687726 22227 catalog_manager.cc:5576] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733) does not have the latest schema for tablet 45a9d5b38fdb44839162b938763e3ad6 (table rebalancer_dynamic_table [id=a4d3a04c126c45e58f86616c2c8010b2]). Expected version 1 got 0
I20240502 07:59:01.689280 27236 tablet.cc:1722] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:01.689421 22227 catalog_manager.cc:5576] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661) does not have the latest schema for tablet 45a9d5b38fdb44839162b938763e3ad6 (table rebalancer_dynamic_table [id=a4d3a04c126c45e58f86616c2c8010b2]). Expected version 1 got 0
I20240502 07:59:01.689821 22222 catalog_manager.cc:5576] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733) does not have the latest schema for tablet 45a9d5b38fdb44839162b938763e3ad6 (table rebalancer_dynamic_table [id=a4d3a04c126c45e58f86616c2c8010b2]). Expected version 1 got 0
I20240502 07:59:01.690690 27056 tablet.cc:1722] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:01.691010 27056 tablet.cc:1620] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:01.691728 27237 tablet.cc:1722] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:01.692108 27237 tablet.cc:1620] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:01.693487 27236 tablet.cc:1620] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:01.693737 22222 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=a4d3a04c126c45e58f86616c2c8010b2] alter complete (version 1)
I20240502 07:59:01.732810 24951 consensus_queue.cc:237] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:01.734175 25083 raft_consensus.cc:1261] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:01.734398 24888 consensus_peers.cc:485] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 -> Peer f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): Couldn't send request to peer f00f3a1876444ad2bf9c919068ab8014. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cba71d2a28984e84b1950d59ec2e5a6a. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:01.734577 24820 raft_consensus.cc:1261] T cba71d2a28984e84b1950d59ec2e5a6a P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:01.734622 27253 consensus_queue.cc:1035] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:01.734977 26758 consensus_queue.cc:1035] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:01.736537 27122 raft_consensus.cc:2932] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:01.737021 24820 raft_consensus.cc:2932] T cba71d2a28984e84b1950d59ec2e5a6a P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:01.739217 25083 raft_consensus.cc:2932] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:01.739882 22229 catalog_manager.cc:5484] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 reported cstate change: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:01.760527 22502 consensus_queue.cc:237] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:01.762037 25214 raft_consensus.cc:1261] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:01.762055 25083 raft_consensus.cc:1261] T f49fcc586e6f498e93723d97b99c077e P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:01.762539 27270 consensus_queue.cc:1035] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 07:59:01.762743 22439 consensus_peers.cc:485] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c -> Peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): Couldn't send request to peer b533a7d8d97943609c6e28e3e36b78a4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f49fcc586e6f498e93723d97b99c077e. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:01.762938 27270 consensus_queue.cc:1035] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:01.764998 26497 raft_consensus.cc:2932] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:01.765398 25214 raft_consensus.cc:2932] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:01.768085 22222 catalog_manager.cc:5484] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c reported cstate change: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:01.769058 25083 raft_consensus.cc:2932] T f49fcc586e6f498e93723d97b99c077e P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:01.789067 22634 consensus_queue.cc:237] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:01.790557 22502 raft_consensus.cc:1261] T 17421eec61584fbcac13f441922269bc P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:01.790863 22567 consensus_peers.cc:485] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f -> Peer 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): Couldn't send request to peer 227eec4b5f014d349661f6e0b39b9055. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 17421eec61584fbcac13f441922269bc. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:01.790896 22764 raft_consensus.cc:1261] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:01.791060 26841 consensus_queue.cc:1035] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:01.791306 27288 consensus_queue.cc:1035] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:01.793177 26841 raft_consensus.cc:2932] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:01.793716 22764 raft_consensus.cc:2932] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:01.795415 22502 raft_consensus.cc:2932] T 17421eec61584fbcac13f441922269bc P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:01.797120 22227 catalog_manager.cc:5484] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:01.804132 22222 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:01.810750 27236 tablet.cc:1722] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:01.811111 27236 tablet.cc:1620] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:01.811264 27056 tablet.cc:1722] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:01.811589 27056 tablet.cc:1620] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:01.813220 27237 tablet.cc:1722] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:01.815605 27237 tablet.cc:1620] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:01.817565 22229 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=a4d3a04c126c45e58f86616c2c8010b2] alter complete (version 2)
I20240502 07:59:01.824836 24951 consensus_queue.cc:237] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:01.826213 25083 raft_consensus.cc:1261] T 4840fa82fcb241f391c4890fb735fe9a P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:01.826267 22371 raft_consensus.cc:1261] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:01.826543 24887 consensus_peers.cc:485] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4840fa82fcb241f391c4890fb735fe9a. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:01.826638 27254 consensus_queue.cc:1035] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:01.826843 26758 consensus_queue.cc:1035] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:01.828881 27254 raft_consensus.cc:2932] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:01.829500 25083 raft_consensus.cc:2932] T 4840fa82fcb241f391c4890fb735fe9a P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:01.831377 22371 raft_consensus.cc:2932] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:01.832309 22227 catalog_manager.cc:5484] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 reported cstate change: config changed from index -1 to 2, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:01.846323 26192 consensus_queue.cc:1035] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:01.849882 27007 consensus_queue.cc:1035] T 751f8c9cbf684b01b90e7bd26685f8bf P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:01.857005 22502 consensus_queue.cc:237] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:01.858613 22764 raft_consensus.cc:1261] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:01.858767 22439 consensus_peers.cc:485] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c -> Peer 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): Couldn't send request to peer 43930dee0cd5498fa8e357a34dd71432. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 847f3a2601d44898a14a0b35409981ad. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:01.858821 22634 raft_consensus.cc:1261] T 847f3a2601d44898a14a0b35409981ad P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:01.859019 26965 consensus_queue.cc:1035] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:01.859305 27271 consensus_queue.cc:1035] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:01.861445 26965 raft_consensus.cc:2932] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:01.861864 22634 raft_consensus.cc:2932] T 847f3a2601d44898a14a0b35409981ad P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:01.864203 22764 raft_consensus.cc:2932] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:01.864866 22222 catalog_manager.cc:5484] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c reported cstate change: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:01.886829 24820 consensus_queue.cc:237] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:01.888360 22764 raft_consensus.cc:1261] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer 4b4e7eeb7f3f42119c0363a79febc54b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:01.888406 25083 raft_consensus.cc:1261] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 4b4e7eeb7f3f42119c0363a79febc54b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:01.888861 24757 consensus_peers.cc:485] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b -> Peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): Couldn't send request to peer b533a7d8d97943609c6e28e3e36b78a4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 01d9a73be1fd472db39860e538b9c97e. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:01.888890 27348 consensus_queue.cc:1035] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:01.889163 27007 consensus_queue.cc:1035] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:01.891106 27348 raft_consensus.cc:2932] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:01.891530 22764 raft_consensus.cc:2932] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:01.893383 25083 raft_consensus.cc:2932] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:01.894318 22229 catalog_manager.cc:5484] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:01.919441 25429 consensus_queue.cc:1035] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:01.921864 22222 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:01.924197 25429 consensus_queue.cc:1035] T a64141e51eb14b04a19cedfa5f346e79 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:01.929992 27236 tablet.cc:1722] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:01.930775 27056 tablet.cc:1722] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:01.931131 27237 tablet.cc:1722] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:01.935832 22226 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=a4d3a04c126c45e58f86616c2c8010b2] alter complete (version 3)
I20240502 07:59:01.986660 25064 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:01.987655 24800 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:01.988312 25195 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:01.994082 24931 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:01.996143 22612 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:01.996634 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:02.011651 22352 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:02.014107 22482 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:02.038367 22222 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:02.038522 22222 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:02.040814 22222 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 45a9d5b38fdb44839162b938763e3ad6
I20240502 07:59:02.041601 22352 tablet_service.cc:1514] Processing DeleteTablet for tablet 45a9d5b38fdb44839162b938763e3ad6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:02 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:02.041874 24800 tablet_service.cc:1514] Processing DeleteTablet for tablet 45a9d5b38fdb44839162b938763e3ad6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:02 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:59:02.042174 27407 tablet_replica.cc:330] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:59:02.042367 27407 raft_consensus.cc:2229] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:02.042575 27407 raft_consensus.cc:2258] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:02.043133 27407 ts_tablet_manager.cc:1875] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:02.043937 25064 tablet_service.cc:1514] Processing DeleteTablet for tablet 45a9d5b38fdb44839162b938763e3ad6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:02 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:02.049589 27407 ts_tablet_manager.cc:1888] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:02.049930 27407 log.cc:1196] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/45a9d5b38fdb44839162b938763e3ad6
I20240502 07:59:02.050658 27407 ts_tablet_manager.cc:1909] T 45a9d5b38fdb44839162b938763e3ad6 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:59:02.051573 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet 45a9d5b38fdb44839162b938763e3ad6 (table rebalancer_dynamic_table_new_name [id=a4d3a04c126c45e58f86616c2c8010b2]) successfully deleted
I20240502 07:59:02.052863 27408 tablet_replica.cc:330] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:02.053015 27408 raft_consensus.cc:2229] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:02.053148 27408 raft_consensus.cc:2258] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:02.054293 27406 tablet_replica.cc:330] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:02.054455 27406 raft_consensus.cc:2229] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:02.054664 27406 raft_consensus.cc:2258] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:02.055222 27406 ts_tablet_manager.cc:1875] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:02.057659 27406 ts_tablet_manager.cc:1888] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:02.057739 27406 log.cc:1196] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/45a9d5b38fdb44839162b938763e3ad6
I20240502 07:59:02.058095 27406 ts_tablet_manager.cc:1909] T 45a9d5b38fdb44839162b938763e3ad6 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:59:02.058743 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 45a9d5b38fdb44839162b938763e3ad6 (table rebalancer_dynamic_table_new_name [id=a4d3a04c126c45e58f86616c2c8010b2]) successfully deleted
I20240502 07:59:02.053908 27408 ts_tablet_manager.cc:1875] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:02.064404 27408 ts_tablet_manager.cc:1888] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:02.064553 27408 log.cc:1196] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/45a9d5b38fdb44839162b938763e3ad6
I20240502 07:59:02.064947 27408 ts_tablet_manager.cc:1909] T 45a9d5b38fdb44839162b938763e3ad6 P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:59:02.065889 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet 45a9d5b38fdb44839162b938763e3ad6 (table rebalancer_dynamic_table_new_name [id=a4d3a04c126c45e58f86616c2c8010b2]) successfully deleted
I20240502 07:59:02.135193 22226 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:02.143090 22482 tablet_service.cc:1467] Processing CreateTablet for tablet 0695ee734e4d494ca64898920ffe9566 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=8d8a3547fe4849369ff81d0455ed07e5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:02.143489 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0695ee734e4d494ca64898920ffe9566. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.143795 24931 tablet_service.cc:1467] Processing CreateTablet for tablet d99bc35db172424bbc8536a1c95601fe (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=8d8a3547fe4849369ff81d0455ed07e5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:02.144068 24931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d99bc35db172424bbc8536a1c95601fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.144124 25064 tablet_service.cc:1467] Processing CreateTablet for tablet d99bc35db172424bbc8536a1c95601fe (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=8d8a3547fe4849369ff81d0455ed07e5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:02.144376 25064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d99bc35db172424bbc8536a1c95601fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.145556 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 0695ee734e4d494ca64898920ffe9566 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=8d8a3547fe4849369ff81d0455ed07e5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:02.145766 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0695ee734e4d494ca64898920ffe9566. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.146026 25195 tablet_service.cc:1467] Processing CreateTablet for tablet 79a67cfa3c554f2d85a520bb8f5ccc81 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=8d8a3547fe4849369ff81d0455ed07e5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:02.146337 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79a67cfa3c554f2d85a520bb8f5ccc81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.150027 27449 tablet_bootstrap.cc:492] T d99bc35db172424bbc8536a1c95601fe P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:02.150511 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 79a67cfa3c554f2d85a520bb8f5ccc81 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=8d8a3547fe4849369ff81d0455ed07e5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:02.150828 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79a67cfa3c554f2d85a520bb8f5ccc81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.151259 27449 tablet_bootstrap.cc:654] T d99bc35db172424bbc8536a1c95601fe P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.151939 27447 tablet_bootstrap.cc:492] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:02.152101 22222 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:02.153169 27447 tablet_bootstrap.cc:654] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.154598 27447 tablet_bootstrap.cc:492] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:02.154697 27447 ts_tablet_manager.cc:1367] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:02.155153 27447 raft_consensus.cc:348] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.155263 27447 raft_consensus.cc:374] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.155297 27447 raft_consensus.cc:729] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.155416 27447 consensus_queue.cc:260] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.155725 27447 ts_tablet_manager.cc:1398] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:02.156126 27447 tablet_bootstrap.cc:492] T 0695ee734e4d494ca64898920ffe9566 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:02.157033 27447 tablet_bootstrap.cc:654] T 0695ee734e4d494ca64898920ffe9566 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.158540 27447 tablet_bootstrap.cc:492] T 0695ee734e4d494ca64898920ffe9566 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:02.158627 27447 ts_tablet_manager.cc:1367] T 0695ee734e4d494ca64898920ffe9566 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:02.159067 27447 raft_consensus.cc:348] T 0695ee734e4d494ca64898920ffe9566 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:02.159164 27447 raft_consensus.cc:374] T 0695ee734e4d494ca64898920ffe9566 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.159196 27447 raft_consensus.cc:729] T 0695ee734e4d494ca64898920ffe9566 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.159302 27447 consensus_queue.cc:260] T 0695ee734e4d494ca64898920ffe9566 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:02.159653 27447 ts_tablet_manager.cc:1398] T 0695ee734e4d494ca64898920ffe9566 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:02.160125 25195 tablet_service.cc:1467] Processing CreateTablet for tablet 7e30c09a2c7b41ea988b84a581b9e5c3 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=928a10a8275b45698bdfdad17f4037a0]), partition=
I20240502 07:59:02.160287 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 7e30c09a2c7b41ea988b84a581b9e5c3 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=928a10a8275b45698bdfdad17f4037a0]), partition=
I20240502 07:59:02.160462 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e30c09a2c7b41ea988b84a581b9e5c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.160524 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e30c09a2c7b41ea988b84a581b9e5c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.161612 27218 consensus_queue.cc:1035] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:02.166045 24800 tablet_service.cc:1467] Processing CreateTablet for tablet d99bc35db172424bbc8536a1c95601fe (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=8d8a3547fe4849369ff81d0455ed07e5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:02.166343 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d99bc35db172424bbc8536a1c95601fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.167142 24799 tablet_service.cc:1467] Processing CreateTablet for tablet 79a67cfa3c554f2d85a520bb8f5ccc81 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=8d8a3547fe4849369ff81d0455ed07e5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:02.167395 24799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79a67cfa3c554f2d85a520bb8f5ccc81. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.167438 26602 consensus_queue.cc:1035] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:02.166888 24801 tablet_service.cc:1467] Processing CreateTablet for tablet 0695ee734e4d494ca64898920ffe9566 (DEFAULT_TABLE table=rebalancer_extra_table_14 [id=8d8a3547fe4849369ff81d0455ed07e5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:02.167840 24801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0695ee734e4d494ca64898920ffe9566. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.169526 24798 tablet_service.cc:1467] Processing CreateTablet for tablet 7e30c09a2c7b41ea988b84a581b9e5c3 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=928a10a8275b45698bdfdad17f4037a0]), partition=
I20240502 07:59:02.169718 24798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e30c09a2c7b41ea988b84a581b9e5c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.176187 27457 tablet_bootstrap.cc:492] T 0695ee734e4d494ca64898920ffe9566 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:02.181767 27448 tablet_bootstrap.cc:492] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:02.182340 27457 tablet_bootstrap.cc:654] T 0695ee734e4d494ca64898920ffe9566 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.185047 27451 tablet_bootstrap.cc:492] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:02.189927 27449 tablet_bootstrap.cc:492] T d99bc35db172424bbc8536a1c95601fe P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:02.190042 27449 ts_tablet_manager.cc:1367] T d99bc35db172424bbc8536a1c95601fe P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.000s
I20240502 07:59:02.190438 27449 raft_consensus.cc:348] T d99bc35db172424bbc8536a1c95601fe P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.190577 27449 raft_consensus.cc:374] T d99bc35db172424bbc8536a1c95601fe P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.190613 27449 raft_consensus.cc:729] T d99bc35db172424bbc8536a1c95601fe P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.190739 27449 consensus_queue.cc:260] T d99bc35db172424bbc8536a1c95601fe P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.191059 27449 ts_tablet_manager.cc:1398] T d99bc35db172424bbc8536a1c95601fe P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:02.195953 27448 tablet_bootstrap.cc:654] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.197306 27451 tablet_bootstrap.cc:654] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.199559 27457 tablet_bootstrap.cc:492] T 0695ee734e4d494ca64898920ffe9566 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:02.199635 27448 tablet_bootstrap.cc:492] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:02.199676 27457 ts_tablet_manager.cc:1367] T 0695ee734e4d494ca64898920ffe9566 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20240502 07:59:02.199818 27448 ts_tablet_manager.cc:1367] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.003s
I20240502 07:59:02.200188 27457 raft_consensus.cc:348] T 0695ee734e4d494ca64898920ffe9566 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:02.200330 27457 raft_consensus.cc:374] T 0695ee734e4d494ca64898920ffe9566 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.200366 27457 raft_consensus.cc:729] T 0695ee734e4d494ca64898920ffe9566 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.200501 27457 consensus_queue.cc:260] T 0695ee734e4d494ca64898920ffe9566 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:02.200806 27457 ts_tablet_manager.cc:1398] T 0695ee734e4d494ca64898920ffe9566 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:02.200994 27457 tablet_bootstrap.cc:492] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:02.200719 27448 raft_consensus.cc:348] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.201881 27448 raft_consensus.cc:374] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.201925 27448 raft_consensus.cc:729] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.202418 27457 tablet_bootstrap.cc:654] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.202431 27448 consensus_queue.cc:260] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.204142 27457 tablet_bootstrap.cc:492] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:02.204241 27457 ts_tablet_manager.cc:1367] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 07:59:02.204684 27448 ts_tablet_manager.cc:1398] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20240502 07:59:02.204736 27457 raft_consensus.cc:348] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.204842 27457 raft_consensus.cc:374] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.204879 27457 raft_consensus.cc:729] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.204888 27448 tablet_bootstrap.cc:492] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:02.205085 27457 consensus_queue.cc:260] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.205433 27457 ts_tablet_manager.cc:1398] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:02.205628 27457 tablet_bootstrap.cc:492] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:02.206065 27448 tablet_bootstrap.cc:654] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.207231 27457 tablet_bootstrap.cc:654] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.211359 27451 tablet_bootstrap.cc:492] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:02.211486 27451 ts_tablet_manager.cc:1367] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.027s user 0.003s sys 0.000s
I20240502 07:59:02.212054 27451 raft_consensus.cc:348] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:02.212193 27451 raft_consensus.cc:374] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.212232 27451 raft_consensus.cc:729] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.212379 27451 consensus_queue.cc:260] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:02.212764 27451 ts_tablet_manager.cc:1398] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:02.212955 27451 tablet_bootstrap.cc:492] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:02.213862 27448 tablet_bootstrap.cc:492] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:02.213958 27448 ts_tablet_manager.cc:1367] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20240502 07:59:02.214228 27451 tablet_bootstrap.cc:654] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.214478 27448 raft_consensus.cc:348] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.214589 27448 raft_consensus.cc:374] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.214725 27448 raft_consensus.cc:729] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.214905 27448 consensus_queue.cc:260] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.215870 27448 ts_tablet_manager.cc:1398] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:02.217182 25064 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:02.218199 24932 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:02.222137 27457 tablet_bootstrap.cc:492] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:02.222380 27457 ts_tablet_manager.cc:1367] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.001s
I20240502 07:59:02.222887 27457 raft_consensus.cc:348] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.223063 27457 raft_consensus.cc:374] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.223136 27457 raft_consensus.cc:729] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.223289 27457 consensus_queue.cc:260] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.224056 27457 ts_tablet_manager.cc:1398] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 07:59:02.224258 27457 tablet_bootstrap.cc:492] T d99bc35db172424bbc8536a1c95601fe P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:02.225244 27457 tablet_bootstrap.cc:654] T d99bc35db172424bbc8536a1c95601fe P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.227619 27457 tablet_bootstrap.cc:492] T d99bc35db172424bbc8536a1c95601fe P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:02.227751 27457 ts_tablet_manager.cc:1367] T d99bc35db172424bbc8536a1c95601fe P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 07:59:02.228220 27457 raft_consensus.cc:348] T d99bc35db172424bbc8536a1c95601fe P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.228382 27457 raft_consensus.cc:374] T d99bc35db172424bbc8536a1c95601fe P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.228448 27457 raft_consensus.cc:729] T d99bc35db172424bbc8536a1c95601fe P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.228628 27457 consensus_queue.cc:260] T d99bc35db172424bbc8536a1c95601fe P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.229027 27457 ts_tablet_manager.cc:1398] T d99bc35db172424bbc8536a1c95601fe P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:02.237198 22352 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:02.252764 24800 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:02.256362 25195 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:02.270592 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:02.273730 26192 raft_consensus.cc:482] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:02.273839 26192 raft_consensus.cc:504] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.277200 22612 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:02.279382 27479 ts_tablet_manager.cc:902] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4: Initiating tablet copy from peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:02.283988 27479 tablet_copy_client.cc:308] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.133:41661
I20240502 07:59:02.284837 24841 tablet_copy_service.cc:140] P 4b4e7eeb7f3f42119c0363a79febc54b: Received BeginTabletCopySession request for tablet 01d9a73be1fd472db39860e538b9c97e from peer b533a7d8d97943609c6e28e3e36b78a4 ({username='slave'} at 127.15.129.129:56569)
I20240502 07:59:02.284976 24841 tablet_copy_service.cc:161] P 4b4e7eeb7f3f42119c0363a79febc54b: Beginning new tablet copy session on tablet 01d9a73be1fd472db39860e538b9c97e from peer b533a7d8d97943609c6e28e3e36b78a4 at {username='slave'} at 127.15.129.129:56569: session id = b533a7d8d97943609c6e28e3e36b78a4-01d9a73be1fd472db39860e538b9c97e
I20240502 07:59:02.286146 27451 tablet_bootstrap.cc:492] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:02.286283 27451 ts_tablet_manager.cc:1367] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.073s user 0.003s sys 0.000s
I20240502 07:59:02.286917 24841 tablet_copy_source_session.cc:205] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:02.286993 27451 raft_consensus.cc:348] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.287142 27451 raft_consensus.cc:374] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.287180 27451 raft_consensus.cc:729] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.287325 27451 consensus_queue.cc:260] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.287851 27007 raft_consensus.cc:482] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:02.287932 27007 raft_consensus.cc:504] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.294011 27451 ts_tablet_manager.cc:1398] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20240502 07:59:02.294397 27479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d9a73be1fd472db39860e538b9c97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.296773 27479 tablet_copy_client.cc:783] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:02.296945 27479 tablet_copy_client.cc:647] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:02.299539 27348 raft_consensus.cc:482] T 0695ee734e4d494ca64898920ffe9566 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:02.299643 27348 raft_consensus.cc:504] T 0695ee734e4d494ca64898920ffe9566 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:02.300431 27451 tablet_bootstrap.cc:492] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:02.300802 27479 tablet_copy_client.cc:523] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:02.302497 27479 tablet_bootstrap.cc:492] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:02.306905 27451 tablet_bootstrap.cc:654] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.307955 26758 raft_consensus.cc:482] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:02.308041 26758 raft_consensus.cc:504] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.308385 26758 leader_election.cc:288] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:02.308744 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d99bc35db172424bbc8536a1c95601fe" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:02.308854 24820 raft_consensus.cc:2454] T d99bc35db172424bbc8536a1c95601fe P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:02.309134 24887 leader_election.cc:302] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:02.309633 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d99bc35db172424bbc8536a1c95601fe" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:02.309753 25083 raft_consensus.cc:2454] T d99bc35db172424bbc8536a1c95601fe P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:02.310293 26758 raft_consensus.cc:2781] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:02.310348 26758 raft_consensus.cc:482] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:02.310379 26758 raft_consensus.cc:3037] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:02.311371 26758 raft_consensus.cc:504] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.311739 26758 leader_election.cc:288] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:02.312189 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d99bc35db172424bbc8536a1c95601fe" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:02.312301 24820 raft_consensus.cc:3037] T d99bc35db172424bbc8536a1c95601fe P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:02.313246 24820 raft_consensus.cc:2454] T d99bc35db172424bbc8536a1c95601fe P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:02.313468 26965 raft_consensus.cc:482] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:02.313601 26965 raft_consensus.cc:504] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:02.313663 24887 leader_election.cc:302] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:02.313974 27253 raft_consensus.cc:2781] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:02.314049 27253 raft_consensus.cc:686] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:02.314174 27253 consensus_queue.cc:237] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.314213 26965 leader_election.cc:288] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:02.316322 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d99bc35db172424bbc8536a1c95601fe" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:02.316448 25083 raft_consensus.cc:3037] T d99bc35db172424bbc8536a1c95601fe P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:02.317392 25083 raft_consensus.cc:2454] T d99bc35db172424bbc8536a1c95601fe P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:02.320880 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0695ee734e4d494ca64898920ffe9566" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:02.321023 24820 raft_consensus.cc:2454] T 0695ee734e4d494ca64898920ffe9566 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:02.321342 22438 leader_election.cc:302] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:02.321588 26965 raft_consensus.cc:2781] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:02.321646 26965 raft_consensus.cc:482] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:02.321683 26965 raft_consensus.cc:3037] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:02.322070 22222 catalog_manager.cc:5484] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:02.322759 26965 raft_consensus.cc:504] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:02.323089 26965 leader_election.cc:288] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:02.326041 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0695ee734e4d494ca64898920ffe9566" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:02.326167 24820 raft_consensus.cc:3037] T 0695ee734e4d494ca64898920ffe9566 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:02.327210 24820 raft_consensus.cc:2454] T 0695ee734e4d494ca64898920ffe9566 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:02.327630 22438 leader_election.cc:302] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:02.327950 27271 raft_consensus.cc:2781] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:02.328037 27271 raft_consensus.cc:686] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:59:02.328176 27271 consensus_queue.cc:237] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:02.328624 22483 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:02.330694 22227 catalog_manager.cc:5484] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:02.333999 27477 ts_tablet_manager.cc:902] T 17421eec61584fbcac13f441922269bc P 227eec4b5f014d349661f6e0b39b9055: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:02.335206 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0695ee734e4d494ca64898920ffe9566" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:02.335352 24951 raft_consensus.cc:2454] T 0695ee734e4d494ca64898920ffe9566 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:02.335608 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0695ee734e4d494ca64898920ffe9566" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:02.335794 24952 raft_consensus.cc:3037] T 0695ee734e4d494ca64898920ffe9566 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:02.337106 27477 tablet_copy_client.cc:308] T 17421eec61584fbcac13f441922269bc P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:59:02.337179 27451 tablet_bootstrap.cc:492] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:02.337289 27451 ts_tablet_manager.cc:1367] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.000s
I20240502 07:59:02.337394 24952 raft_consensus.cc:2454] T 0695ee734e4d494ca64898920ffe9566 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:02.337661 22653 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet 17421eec61584fbcac13f441922269bc from peer 227eec4b5f014d349661f6e0b39b9055 ({username='slave'} at 127.15.129.135:39233)
I20240502 07:59:02.337756 22653 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet 17421eec61584fbcac13f441922269bc from peer 227eec4b5f014d349661f6e0b39b9055 at {username='slave'} at 127.15.129.135:39233: session id = 227eec4b5f014d349661f6e0b39b9055-17421eec61584fbcac13f441922269bc
I20240502 07:59:02.337934 27451 raft_consensus.cc:348] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.338189 27451 raft_consensus.cc:374] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.338258 27451 raft_consensus.cc:729] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.338429 27451 consensus_queue.cc:260] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.338706 22653 tablet_copy_source_session.cc:205] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:02.338829 27451 ts_tablet_manager.cc:1398] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:02.339558 27477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17421eec61584fbcac13f441922269bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.341516 27478 ts_tablet_manager.cc:902] T 4840fa82fcb241f391c4890fb735fe9a P 4b4e7eeb7f3f42119c0363a79febc54b: Initiating tablet copy from peer 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:02.344682 27007 leader_election.cc:288] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:02.345182 27348 leader_election.cc:288] T 0695ee734e4d494ca64898920ffe9566 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:02.345593 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0695ee734e4d494ca64898920ffe9566" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:02.345779 24952 raft_consensus.cc:2379] T 0695ee734e4d494ca64898920ffe9566 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b4e7eeb7f3f42119c0363a79febc54b in current term 1: Already voted for candidate fabddb16c85944648a710ec52fc7244c in this term.
I20240502 07:59:02.345824 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e30c09a2c7b41ea988b84a581b9e5c3" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:02.345957 25214 raft_consensus.cc:2454] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:02.345986 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79a67cfa3c554f2d85a520bb8f5ccc81" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:02.346072 25215 raft_consensus.cc:2454] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:02.344687 26192 leader_election.cc:288] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:02.346381 24757 leader_election.cc:302] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:02.346686 24757 leader_election.cc:302] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:02.346969 27348 raft_consensus.cc:2781] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:02.347028 27348 raft_consensus.cc:482] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:02.347070 27348 raft_consensus.cc:3037] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:02.348166 27348 raft_consensus.cc:504] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.348564 27348 leader_election.cc:288] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:02.348882 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79a67cfa3c554f2d85a520bb8f5ccc81" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:02.346699 26192 raft_consensus.cc:2781] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:02.349012 26192 raft_consensus.cc:482] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:02.349011 25215 raft_consensus.cc:3037] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:02.349059 26192 raft_consensus.cc:3037] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:02.350073 25215 raft_consensus.cc:2454] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:02.350199 26192 raft_consensus.cc:504] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.350580 24757 leader_election.cc:302] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:02.350630 26192 leader_election.cc:288] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:02.350805 27007 raft_consensus.cc:2781] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:02.350896 27007 raft_consensus.cc:686] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:59:02.351045 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e30c09a2c7b41ea988b84a581b9e5c3" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:02.351037 27007 consensus_queue.cc:237] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.351143 25215 raft_consensus.cc:3037] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:02.352048 25215 raft_consensus.cc:2454] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:02.352514 24757 leader_election.cc:302] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:02.352741 27007 raft_consensus.cc:2781] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:02.352810 27007 raft_consensus.cc:686] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:59:02.352927 27007 consensus_queue.cc:237] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.354748 22227 catalog_manager.cc:5484] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:02.355466 27478 tablet_copy_client.cc:308] T 4840fa82fcb241f391c4890fb735fe9a P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.134:35995
I20240502 07:59:02.356169 24972 tablet_copy_service.cc:140] P 1aecef81733c47b3b468504934357231: Received BeginTabletCopySession request for tablet 4840fa82fcb241f391c4890fb735fe9a from peer 4b4e7eeb7f3f42119c0363a79febc54b ({username='slave'} at 127.15.129.133:58103)
I20240502 07:59:02.356305 24972 tablet_copy_service.cc:161] P 1aecef81733c47b3b468504934357231: Beginning new tablet copy session on tablet 4840fa82fcb241f391c4890fb735fe9a from peer 4b4e7eeb7f3f42119c0363a79febc54b at {username='slave'} at 127.15.129.133:58103: session id = 4b4e7eeb7f3f42119c0363a79febc54b-4840fa82fcb241f391c4890fb735fe9a
I20240502 07:59:02.357859 24972 tablet_copy_source_session.cc:205] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:02.358844 22227 catalog_manager.cc:5484] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:02.361711 27478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4840fa82fcb241f391c4890fb735fe9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.363868 27478 tablet_copy_client.cc:783] T 4840fa82fcb241f391c4890fb735fe9a P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:02.364154 27478 tablet_copy_client.cc:647] T 4840fa82fcb241f391c4890fb735fe9a P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:02.364406 27484 ts_tablet_manager.cc:902] T cba71d2a28984e84b1950d59ec2e5a6a P f00f3a1876444ad2bf9c919068ab8014: Initiating tablet copy from peer 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:02.365257 27484 tablet_copy_client.cc:308] T cba71d2a28984e84b1950d59ec2e5a6a P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.134:35995
I20240502 07:59:02.366972 24972 tablet_copy_service.cc:140] P 1aecef81733c47b3b468504934357231: Received BeginTabletCopySession request for tablet cba71d2a28984e84b1950d59ec2e5a6a from peer f00f3a1876444ad2bf9c919068ab8014 ({username='slave'} at 127.15.129.132:59331)
I20240502 07:59:02.367097 24972 tablet_copy_service.cc:161] P 1aecef81733c47b3b468504934357231: Beginning new tablet copy session on tablet cba71d2a28984e84b1950d59ec2e5a6a from peer f00f3a1876444ad2bf9c919068ab8014 at {username='slave'} at 127.15.129.132:59331: session id = f00f3a1876444ad2bf9c919068ab8014-cba71d2a28984e84b1950d59ec2e5a6a
I20240502 07:59:02.367961 27479 tablet_bootstrap.cc:492] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:02.368204 24972 tablet_copy_source_session.cc:205] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:02.368254 27479 tablet_bootstrap.cc:492] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap complete.
I20240502 07:59:02.368866 27479 ts_tablet_manager.cc:1367] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.066s user 0.006s sys 0.000s
I20240502 07:59:02.369508 27479 raft_consensus.cc:348] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:02.369628 27479 raft_consensus.cc:729] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.369791 27479 consensus_queue.cc:260] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:02.370288 27479 ts_tablet_manager.cc:1398] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:02.371018 24841 tablet_copy_service.cc:342] P 4b4e7eeb7f3f42119c0363a79febc54b: Request end of tablet copy session b533a7d8d97943609c6e28e3e36b78a4-01d9a73be1fd472db39860e538b9c97e received from {username='slave'} at 127.15.129.129:56569
I20240502 07:59:02.371102 24841 tablet_copy_service.cc:433] P 4b4e7eeb7f3f42119c0363a79febc54b: ending tablet copy session b533a7d8d97943609c6e28e3e36b78a4-01d9a73be1fd472db39860e538b9c97e on tablet 01d9a73be1fd472db39860e538b9c97e with peer b533a7d8d97943609c6e28e3e36b78a4
I20240502 07:59:02.371974 27491 ts_tablet_manager.cc:902] T f49fcc586e6f498e93723d97b99c077e P b533a7d8d97943609c6e28e3e36b78a4: Initiating tablet copy from peer fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:02.372797 27491 tablet_copy_client.cc:308] T f49fcc586e6f498e93723d97b99c077e P b533a7d8d97943609c6e28e3e36b78a4: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.130:39625
I20240502 07:59:02.373396 22523 tablet_copy_service.cc:140] P fabddb16c85944648a710ec52fc7244c: Received BeginTabletCopySession request for tablet f49fcc586e6f498e93723d97b99c077e from peer b533a7d8d97943609c6e28e3e36b78a4 ({username='slave'} at 127.15.129.129:34277)
I20240502 07:59:02.373500 22523 tablet_copy_service.cc:161] P fabddb16c85944648a710ec52fc7244c: Beginning new tablet copy session on tablet f49fcc586e6f498e93723d97b99c077e from peer b533a7d8d97943609c6e28e3e36b78a4 at {username='slave'} at 127.15.129.129:34277: session id = b533a7d8d97943609c6e28e3e36b78a4-f49fcc586e6f498e93723d97b99c077e
I20240502 07:59:02.374433 22523 tablet_copy_source_session.cc:205] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:02.376418 27477 tablet_copy_client.cc:783] T 17421eec61584fbcac13f441922269bc P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:02.376605 27477 tablet_copy_client.cc:647] T 17421eec61584fbcac13f441922269bc P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:02.377148 27491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f49fcc586e6f498e93723d97b99c077e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.377897 27477 tablet_copy_client.cc:523] T 17421eec61584fbcac13f441922269bc P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:02.378986 27477 tablet_bootstrap.cc:492] T 17421eec61584fbcac13f441922269bc P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:02.379292 27491 tablet_copy_client.cc:783] T f49fcc586e6f498e93723d97b99c077e P b533a7d8d97943609c6e28e3e36b78a4: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:02.379451 27491 tablet_copy_client.cc:647] T f49fcc586e6f498e93723d97b99c077e P b533a7d8d97943609c6e28e3e36b78a4: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:02.380086 27484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cba71d2a28984e84b1950d59ec2e5a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.381389 27478 tablet_copy_client.cc:523] T 4840fa82fcb241f391c4890fb735fe9a P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:02.382505 27491 tablet_copy_client.cc:523] T f49fcc586e6f498e93723d97b99c077e P b533a7d8d97943609c6e28e3e36b78a4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:02.382896 27478 tablet_bootstrap.cc:492] T 4840fa82fcb241f391c4890fb735fe9a P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:02.383860 27491 tablet_bootstrap.cc:492] T f49fcc586e6f498e93723d97b99c077e P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:02.387813 27491 tablet_bootstrap.cc:492] T f49fcc586e6f498e93723d97b99c077e P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:02.388051 27491 tablet_bootstrap.cc:492] T f49fcc586e6f498e93723d97b99c077e P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap complete.
I20240502 07:59:02.388188 27491 ts_tablet_manager.cc:1367] T f49fcc586e6f498e93723d97b99c077e P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 07:59:02.388629 27491 raft_consensus.cc:348] T f49fcc586e6f498e93723d97b99c077e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:02.388751 27491 raft_consensus.cc:729] T f49fcc586e6f498e93723d97b99c077e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.388864 27491 consensus_queue.cc:260] T f49fcc586e6f498e93723d97b99c077e P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:02.389128 27491 ts_tablet_manager.cc:1398] T f49fcc586e6f498e93723d97b99c077e P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:02.390470 22523 tablet_copy_service.cc:342] P fabddb16c85944648a710ec52fc7244c: Request end of tablet copy session b533a7d8d97943609c6e28e3e36b78a4-f49fcc586e6f498e93723d97b99c077e received from {username='slave'} at 127.15.129.129:34277
I20240502 07:59:02.390578 22523 tablet_copy_service.cc:433] P fabddb16c85944648a710ec52fc7244c: ending tablet copy session b533a7d8d97943609c6e28e3e36b78a4-f49fcc586e6f498e93723d97b99c077e on tablet f49fcc586e6f498e93723d97b99c077e with peer b533a7d8d97943609c6e28e3e36b78a4
I20240502 07:59:02.393239 27484 tablet_copy_client.cc:783] T cba71d2a28984e84b1950d59ec2e5a6a P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:02.393563 27484 tablet_copy_client.cc:647] T cba71d2a28984e84b1950d59ec2e5a6a P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:02.395032 27484 tablet_copy_client.cc:523] T cba71d2a28984e84b1950d59ec2e5a6a P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:02.396853 27484 tablet_bootstrap.cc:492] T cba71d2a28984e84b1950d59ec2e5a6a P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:02.397419 27508 ts_tablet_manager.cc:902] T 847f3a2601d44898a14a0b35409981ad P 43930dee0cd5498fa8e357a34dd71432: Initiating tablet copy from peer fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:02.401975 27478 tablet_bootstrap.cc:492] T 4840fa82fcb241f391c4890fb735fe9a P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:02.402212 27508 tablet_copy_client.cc:308] T 847f3a2601d44898a14a0b35409981ad P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.130:39625
I20240502 07:59:02.402691 27478 tablet_bootstrap.cc:492] T 4840fa82fcb241f391c4890fb735fe9a P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap complete.
I20240502 07:59:02.402964 27478 ts_tablet_manager.cc:1367] T 4840fa82fcb241f391c4890fb735fe9a P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.000s
I20240502 07:59:02.402940 22523 tablet_copy_service.cc:140] P fabddb16c85944648a710ec52fc7244c: Received BeginTabletCopySession request for tablet 847f3a2601d44898a14a0b35409981ad from peer 43930dee0cd5498fa8e357a34dd71432 ({username='slave'} at 127.15.129.136:45143)
I20240502 07:59:02.403035 22523 tablet_copy_service.cc:161] P fabddb16c85944648a710ec52fc7244c: Beginning new tablet copy session on tablet 847f3a2601d44898a14a0b35409981ad from peer 43930dee0cd5498fa8e357a34dd71432 at {username='slave'} at 127.15.129.136:45143: session id = 43930dee0cd5498fa8e357a34dd71432-847f3a2601d44898a14a0b35409981ad
I20240502 07:59:02.403541 27478 raft_consensus.cc:348] T 4840fa82fcb241f391c4890fb735fe9a P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.403748 27478 raft_consensus.cc:729] T 4840fa82fcb241f391c4890fb735fe9a P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.404053 22523 tablet_copy_source_session.cc:205] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:02.404208 27478 consensus_queue.cc:260] T 4840fa82fcb241f391c4890fb735fe9a P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.404301 27477 tablet_bootstrap.cc:492] T 17421eec61584fbcac13f441922269bc P 227eec4b5f014d349661f6e0b39b9055: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:02.404606 27478 ts_tablet_manager.cc:1398] T 4840fa82fcb241f391c4890fb735fe9a P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:02.404824 27477 tablet_bootstrap.cc:492] T 17421eec61584fbcac13f441922269bc P 227eec4b5f014d349661f6e0b39b9055: Bootstrap complete.
I20240502 07:59:02.405030 27477 ts_tablet_manager.cc:1367] T 17421eec61584fbcac13f441922269bc P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.026s user 0.025s sys 0.000s
I20240502 07:59:02.405095 22502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e30c09a2c7b41ea988b84a581b9e5c3" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:02.405218 22502 raft_consensus.cc:2454] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:02.405287 22500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79a67cfa3c554f2d85a520bb8f5ccc81" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:02.405439 27508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 847f3a2601d44898a14a0b35409981ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.405437 22503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79a67cfa3c554f2d85a520bb8f5ccc81" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:02.405550 22499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e30c09a2c7b41ea988b84a581b9e5c3" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:02.405589 27477 raft_consensus.cc:348] T 17421eec61584fbcac13f441922269bc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.405644 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0695ee734e4d494ca64898920ffe9566" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:02.405691 27477 raft_consensus.cc:729] T 17421eec61584fbcac13f441922269bc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.405830 24972 tablet_copy_service.cc:342] P 1aecef81733c47b3b468504934357231: Request end of tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-4840fa82fcb241f391c4890fb735fe9a received from {username='slave'} at 127.15.129.133:58103
I20240502 07:59:02.405846 27477 consensus_queue.cc:260] T 17421eec61584fbcac13f441922269bc P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.405921 24972 tablet_copy_service.cc:433] P 1aecef81733c47b3b468504934357231: ending tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-4840fa82fcb241f391c4890fb735fe9a on tablet 4840fa82fcb241f391c4890fb735fe9a with peer 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:02.406203 27477 ts_tablet_manager.cc:1398] T 17421eec61584fbcac13f441922269bc P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:02.405652 22499 raft_consensus.cc:3037] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:02.406752 22653 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session 227eec4b5f014d349661f6e0b39b9055-17421eec61584fbcac13f441922269bc received from {username='slave'} at 127.15.129.135:39233
I20240502 07:59:02.406821 22653 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session 227eec4b5f014d349661f6e0b39b9055-17421eec61584fbcac13f441922269bc on tablet 17421eec61584fbcac13f441922269bc with peer 227eec4b5f014d349661f6e0b39b9055
I20240502 07:59:02.407516 22499 raft_consensus.cc:2454] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:02.405447 22500 raft_consensus.cc:3037] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:02.408048 24754 leader_election.cc:302] T 0695ee734e4d494ca64898920ffe9566 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b; no voters: 1aecef81733c47b3b468504934357231, fabddb16c85944648a710ec52fc7244c
I20240502 07:59:02.408224 27496 raft_consensus.cc:2726] T 0695ee734e4d494ca64898920ffe9566 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 07:59:02.408533 27508 tablet_copy_client.cc:783] T 847f3a2601d44898a14a0b35409981ad P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:02.408687 27508 tablet_copy_client.cc:647] T 847f3a2601d44898a14a0b35409981ad P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:02.408804 22500 raft_consensus.cc:2454] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:02.411237 27508 tablet_copy_client.cc:523] T 847f3a2601d44898a14a0b35409981ad P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:02.412483 27508 tablet_bootstrap.cc:492] T 847f3a2601d44898a14a0b35409981ad P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:02.416355 27484 tablet_bootstrap.cc:492] T cba71d2a28984e84b1950d59ec2e5a6a P f00f3a1876444ad2bf9c919068ab8014: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:02.416718 27484 tablet_bootstrap.cc:492] T cba71d2a28984e84b1950d59ec2e5a6a P f00f3a1876444ad2bf9c919068ab8014: Bootstrap complete.
I20240502 07:59:02.416913 27484 ts_tablet_manager.cc:1367] T cba71d2a28984e84b1950d59ec2e5a6a P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.001s
I20240502 07:59:02.417884 27484 raft_consensus.cc:348] T cba71d2a28984e84b1950d59ec2e5a6a P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:02.417997 27484 raft_consensus.cc:729] T cba71d2a28984e84b1950d59ec2e5a6a P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.418169 27484 consensus_queue.cc:260] T cba71d2a28984e84b1950d59ec2e5a6a P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:02.418464 27484 ts_tablet_manager.cc:1398] T cba71d2a28984e84b1950d59ec2e5a6a P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240502 07:59:02.419234 24972 tablet_copy_service.cc:342] P 1aecef81733c47b3b468504934357231: Request end of tablet copy session f00f3a1876444ad2bf9c919068ab8014-cba71d2a28984e84b1950d59ec2e5a6a received from {username='slave'} at 127.15.129.132:59331
I20240502 07:59:02.419315 24972 tablet_copy_service.cc:433] P 1aecef81733c47b3b468504934357231: ending tablet copy session f00f3a1876444ad2bf9c919068ab8014-cba71d2a28984e84b1950d59ec2e5a6a on tablet cba71d2a28984e84b1950d59ec2e5a6a with peer f00f3a1876444ad2bf9c919068ab8014
I20240502 07:59:02.440490 27508 tablet_bootstrap.cc:492] T 847f3a2601d44898a14a0b35409981ad P 43930dee0cd5498fa8e357a34dd71432: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:02.440827 27508 tablet_bootstrap.cc:492] T 847f3a2601d44898a14a0b35409981ad P 43930dee0cd5498fa8e357a34dd71432: Bootstrap complete.
I20240502 07:59:02.441001 27508 ts_tablet_manager.cc:1367] T 847f3a2601d44898a14a0b35409981ad P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.029s user 0.021s sys 0.002s
I20240502 07:59:02.441498 27508 raft_consensus.cc:348] T 847f3a2601d44898a14a0b35409981ad P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:02.441639 27508 raft_consensus.cc:729] T 847f3a2601d44898a14a0b35409981ad P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.441823 27508 consensus_queue.cc:260] T 847f3a2601d44898a14a0b35409981ad P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:02.442119 27508 ts_tablet_manager.cc:1398] T 847f3a2601d44898a14a0b35409981ad P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.005s sys 0.000s
I20240502 07:59:02.442790 22523 tablet_copy_service.cc:342] P fabddb16c85944648a710ec52fc7244c: Request end of tablet copy session 43930dee0cd5498fa8e357a34dd71432-847f3a2601d44898a14a0b35409981ad received from {username='slave'} at 127.15.129.136:45143
I20240502 07:59:02.442884 22523 tablet_copy_service.cc:433] P fabddb16c85944648a710ec52fc7244c: ending tablet copy session 43930dee0cd5498fa8e357a34dd71432-847f3a2601d44898a14a0b35409981ad on tablet 847f3a2601d44898a14a0b35409981ad with peer 43930dee0cd5498fa8e357a34dd71432
I20240502 07:59:02.473731 22229 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:02.479102 25215 raft_consensus.cc:1261] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 4b4e7eeb7f3f42119c0363a79febc54b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:02.479139 22500 raft_consensus.cc:1261] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer 4b4e7eeb7f3f42119c0363a79febc54b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:02.480758 26192 consensus_queue.cc:1035] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:02.481369 26192 consensus_queue.cc:1035] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:02.484876 22225 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 7e30c09a2c7b41ea988b84a581b9e5c3 (table rebalancer_dynamic_table [id=928a10a8275b45698bdfdad17f4037a0]). Expected version 1 got 0
I20240502 07:59:02.485736 22227 catalog_manager.cc:5576] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625) does not have the latest schema for tablet 7e30c09a2c7b41ea988b84a581b9e5c3 (table rebalancer_dynamic_table [id=928a10a8275b45698bdfdad17f4037a0]). Expected version 1 got 0
I20240502 07:59:02.494592 27563 mvcc.cc:205] Tried to move back new op lower bound from 7023152097187782656 to 7023152096678879232. Current Snapshot: MvccSnapshot[applied={T|T < 7023152097187782656}]
I20240502 07:59:02.495601 22227 catalog_manager.cc:5576] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625) does not have the latest schema for tablet 7e30c09a2c7b41ea988b84a581b9e5c3 (table rebalancer_dynamic_table [id=928a10a8275b45698bdfdad17f4037a0]). Expected version 1 got 0
I20240502 07:59:02.495939 27566 tablet.cc:1722] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:02.496037 22229 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 7e30c09a2c7b41ea988b84a581b9e5c3 (table rebalancer_dynamic_table [id=928a10a8275b45698bdfdad17f4037a0]). Expected version 1 got 0
I20240502 07:59:02.496337 27566 tablet.cc:1620] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:02.497052 27567 tablet.cc:1722] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:02.500084 27567 tablet.cc:1620] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c: MemRowSet was empty: no flush needed.
I20240502 07:59:02.505071 27568 tablet.cc:1722] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:02.505455 27568 tablet.cc:1620] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:02.505611 22224 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=928a10a8275b45698bdfdad17f4037a0] alter complete (version 1)
I20240502 07:59:02.522131 24932 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:02.522729 25064 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:02.523298 24800 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:02.536130 25195 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:02.536772 22612 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:02.543108 22352 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:02.561442 22483 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:02.570399 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:02.608690 22226 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:02.614478 27566 tablet.cc:1722] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:02.614853 27566 tablet.cc:1620] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:02.615506 27568 tablet.cc:1722] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:02.615818 27568 tablet.cc:1620] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:02.616103 27567 tablet.cc:1722] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:02.616423 27567 tablet.cc:1620] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c: MemRowSet was empty: no flush needed.
I20240502 07:59:02.621141 22224 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=928a10a8275b45698bdfdad17f4037a0] alter complete (version 2)
I20240502 07:59:02.691774 24932 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:02.699616 22612 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:02.702592 25064 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:02.702847 22483 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:02.706555 24800 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:02.716168 22371 raft_consensus.cc:1203] T f49fcc586e6f498e93723d97b99c077e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:02.717593 22352 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:02.718386 25195 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:02.726384 22224 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:02.739646 27566 tablet.cc:1722] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:02.742179 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:02.742424 27567 tablet.cc:1722] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:02.743893 27568 tablet.cc:1722] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:02.746868 22224 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=928a10a8275b45698bdfdad17f4037a0] alter complete (version 3)
I20240502 07:59:02.747109 22371 raft_consensus.cc:1203] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:02.772567 26965 consensus_queue.cc:1035] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:02.775415 22764 raft_consensus.cc:1203] T cba71d2a28984e84b1950d59ec2e5a6a P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:02.782662 26965 consensus_queue.cc:1035] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:02.785840 25083 raft_consensus.cc:1203] T 17421eec61584fbcac13f441922269bc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:02.811509 24820 raft_consensus.cc:1203] T 4840fa82fcb241f391c4890fb735fe9a P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:02.821429 26758 consensus_queue.cc:1035] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:02.824671 25215 raft_consensus.cc:1203] T 847f3a2601d44898a14a0b35409981ad P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:02.825208 27254 consensus_queue.cc:1035] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:02.851828 22221 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:02.852049 22221 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:02.854296 22221 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 7e30c09a2c7b41ea988b84a581b9e5c3
I20240502 07:59:02.854924 25195 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e30c09a2c7b41ea988b84a581b9e5c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:02 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:02.855664 24800 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e30c09a2c7b41ea988b84a581b9e5c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:02 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:59:02.855937 27678 tablet_replica.cc:330] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:02.856132 27678 raft_consensus.cc:2229] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:02.856313 27678 raft_consensus.cc:2258] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:02.856343 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e30c09a2c7b41ea988b84a581b9e5c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:02 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:02.856994 27678 ts_tablet_manager.cc:1875] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:02.857492 27680 tablet_replica.cc:330] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:02.857520 27679 tablet_replica.cc:330] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:59:02.857645 27679 raft_consensus.cc:2229] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:02.857738 27680 raft_consensus.cc:2229] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:02.857872 27679 raft_consensus.cc:2258] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:02.857897 27680 raft_consensus.cc:2258] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:02.858439 27680 ts_tablet_manager.cc:1875] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:02.858597 27679 ts_tablet_manager.cc:1875] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:02.860627 27680 ts_tablet_manager.cc:1888] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:02.860627 27679 ts_tablet_manager.cc:1888] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:02.860702 27679 log.cc:1196] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/7e30c09a2c7b41ea988b84a581b9e5c3
I20240502 07:59:02.860802 27680 log.cc:1196] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/7e30c09a2c7b41ea988b84a581b9e5c3
I20240502 07:59:02.861045 27679 ts_tablet_manager.cc:1909] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:59:02.861194 27680 ts_tablet_manager.cc:1909] T 7e30c09a2c7b41ea988b84a581b9e5c3 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:59:02.861883 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 7e30c09a2c7b41ea988b84a581b9e5c3 (table rebalancer_dynamic_table_new_name [id=928a10a8275b45698bdfdad17f4037a0]) successfully deleted
I20240502 07:59:02.862366 27678 ts_tablet_manager.cc:1888] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:02.862448 27678 log.cc:1196] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/7e30c09a2c7b41ea988b84a581b9e5c3
I20240502 07:59:02.862761 27678 ts_tablet_manager.cc:1909] T 7e30c09a2c7b41ea988b84a581b9e5c3 P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:59:02.862996 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet 7e30c09a2c7b41ea988b84a581b9e5c3 (table rebalancer_dynamic_table_new_name [id=928a10a8275b45698bdfdad17f4037a0]) successfully deleted
I20240502 07:59:02.863687 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 7e30c09a2c7b41ea988b84a581b9e5c3 (table rebalancer_dynamic_table_new_name [id=928a10a8275b45698bdfdad17f4037a0]) successfully deleted
I20240502 07:59:02.866854 24932 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:02.868059 22221 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_16"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:02.881263 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 4423a78b96cd42f8a509dc1f28d87a7f (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=9e1975d2adc44bd9979e036fbd9b62ca]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:02.881263 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 5d342c368796426c81bede7ddc23937b (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=9e1975d2adc44bd9979e036fbd9b62ca]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:02.881583 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d342c368796426c81bede7ddc23937b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.881587 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4423a78b96cd42f8a509dc1f28d87a7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.881798 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 4423a78b96cd42f8a509dc1f28d87a7f (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=9e1975d2adc44bd9979e036fbd9b62ca]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:02.882055 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4423a78b96cd42f8a509dc1f28d87a7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.883808 27692 tablet_bootstrap.cc:492] T 4423a78b96cd42f8a509dc1f28d87a7f P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:02.884104 24799 tablet_service.cc:1467] Processing CreateTablet for tablet 712a88cd545745e8937d295266e85f04 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=9e1975d2adc44bd9979e036fbd9b62ca]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:02.884253 27693 tablet_bootstrap.cc:492] T 5d342c368796426c81bede7ddc23937b P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:02.884337 24799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 712a88cd545745e8937d295266e85f04. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.885288 25194 tablet_service.cc:1467] Processing CreateTablet for tablet 712a88cd545745e8937d295266e85f04 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=9e1975d2adc44bd9979e036fbd9b62ca]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:02.885615 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 712a88cd545745e8937d295266e85f04. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.886487 25195 tablet_service.cc:1467] Processing CreateTablet for tablet 5d342c368796426c81bede7ddc23937b (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=9e1975d2adc44bd9979e036fbd9b62ca]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:02.886734 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d342c368796426c81bede7ddc23937b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.888038 25064 tablet_service.cc:1467] Processing CreateTablet for tablet 4423a78b96cd42f8a509dc1f28d87a7f (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=9e1975d2adc44bd9979e036fbd9b62ca]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:02.888135 22352 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:02.888284 25064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4423a78b96cd42f8a509dc1f28d87a7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.888866 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 5d342c368796426c81bede7ddc23937b (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=9e1975d2adc44bd9979e036fbd9b62ca]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:02.889071 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d342c368796426c81bede7ddc23937b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.891109 25062 tablet_service.cc:1467] Processing CreateTablet for tablet 712a88cd545745e8937d295266e85f04 (DEFAULT_TABLE table=rebalancer_extra_table_16 [id=9e1975d2adc44bd9979e036fbd9b62ca]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:02.891300 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 712a88cd545745e8937d295266e85f04. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.894121 24799 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:02.895383 27696 tablet_bootstrap.cc:492] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:02.895643 27695 tablet_bootstrap.cc:492] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:02.895821 27693 tablet_bootstrap.cc:654] T 5d342c368796426c81bede7ddc23937b P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.896819 27692 tablet_bootstrap.cc:654] T 4423a78b96cd42f8a509dc1f28d87a7f P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.899459 22612 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:02.900496 25195 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:02.900986 22483 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:02.902943 27696 tablet_bootstrap.cc:654] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.902988 25062 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:02.904886 27695 tablet_bootstrap.cc:654] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.905741 26192 consensus_queue.cc:1035] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:02.906114 27696 tablet_bootstrap.cc:492] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:02.906235 27696 ts_tablet_manager.cc:1367] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 07:59:02.906724 27696 raft_consensus.cc:348] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.906978 27696 raft_consensus.cc:374] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.907068 27696 raft_consensus.cc:729] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.907307 27696 consensus_queue.cc:260] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.907719 27696 ts_tablet_manager.cc:1398] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:02.907928 27696 tablet_bootstrap.cc:492] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:02.919206 27696 tablet_bootstrap.cc:654] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.923018 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:02.926080 27695 tablet_bootstrap.cc:492] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:02.926242 27695 ts_tablet_manager.cc:1367] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.000s
I20240502 07:59:02.926913 27695 raft_consensus.cc:348] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.927227 27695 raft_consensus.cc:374] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.927309 27695 raft_consensus.cc:729] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.927484 27695 consensus_queue.cc:260] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.928388 27695 ts_tablet_manager.cc:1398] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:02.928586 27695 tablet_bootstrap.cc:492] T 5d342c368796426c81bede7ddc23937b P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:02.929602 27695 tablet_bootstrap.cc:654] T 5d342c368796426c81bede7ddc23937b P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.931362 27694 tablet_bootstrap.cc:492] T 4423a78b96cd42f8a509dc1f28d87a7f P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:02.932605 27696 tablet_bootstrap.cc:492] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:02.932612 27694 tablet_bootstrap.cc:654] T 4423a78b96cd42f8a509dc1f28d87a7f P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.932760 27696 ts_tablet_manager.cc:1367] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20240502 07:59:02.933228 27696 raft_consensus.cc:348] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.933467 27696 raft_consensus.cc:374] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.933537 27696 raft_consensus.cc:729] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.933712 27696 consensus_queue.cc:260] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.934098 27696 ts_tablet_manager.cc:1398] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:02.934285 27694 tablet_bootstrap.cc:492] T 4423a78b96cd42f8a509dc1f28d87a7f P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:02.934288 27696 tablet_bootstrap.cc:492] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:02.934434 27694 ts_tablet_manager.cc:1367] T 4423a78b96cd42f8a509dc1f28d87a7f P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:02.934876 27694 raft_consensus.cc:348] T 4423a78b96cd42f8a509dc1f28d87a7f P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.935035 27694 raft_consensus.cc:374] T 4423a78b96cd42f8a509dc1f28d87a7f P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.935101 27694 raft_consensus.cc:729] T 4423a78b96cd42f8a509dc1f28d87a7f P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.935278 27694 consensus_queue.cc:260] T 4423a78b96cd42f8a509dc1f28d87a7f P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.935703 27694 ts_tablet_manager.cc:1398] T 4423a78b96cd42f8a509dc1f28d87a7f P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:02.936450 27694 tablet_bootstrap.cc:492] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:02.938683 27694 tablet_bootstrap.cc:654] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.939955 27694 tablet_bootstrap.cc:492] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:02.940028 27694 ts_tablet_manager.cc:1367] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:02.940425 27696 tablet_bootstrap.cc:654] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.940423 27694 raft_consensus.cc:348] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.940515 27694 raft_consensus.cc:374] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.940541 27694 raft_consensus.cc:729] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.940639 27694 consensus_queue.cc:260] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.940866 27694 ts_tablet_manager.cc:1398] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:02.941514 27695 tablet_bootstrap.cc:492] T 5d342c368796426c81bede7ddc23937b P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:02.941613 27695 ts_tablet_manager.cc:1367] T 5d342c368796426c81bede7ddc23937b P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 07:59:02.942039 27695 raft_consensus.cc:348] T 5d342c368796426c81bede7ddc23937b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.942132 27695 raft_consensus.cc:374] T 5d342c368796426c81bede7ddc23937b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.942160 27695 raft_consensus.cc:729] T 5d342c368796426c81bede7ddc23937b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.942270 27695 consensus_queue.cc:260] T 5d342c368796426c81bede7ddc23937b P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.942523 27695 ts_tablet_manager.cc:1398] T 5d342c368796426c81bede7ddc23937b P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:02.945955 27696 tablet_bootstrap.cc:492] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:02.946051 27696 ts_tablet_manager.cc:1367] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 07:59:02.946504 27696 raft_consensus.cc:348] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.946605 27696 raft_consensus.cc:374] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.946632 27696 raft_consensus.cc:729] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.946892 27696 consensus_queue.cc:260] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.947233 27696 ts_tablet_manager.cc:1398] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:02.949214 27693 tablet_bootstrap.cc:492] T 5d342c368796426c81bede7ddc23937b P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:02.949795 27692 tablet_bootstrap.cc:492] T 4423a78b96cd42f8a509dc1f28d87a7f P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:02.949880 27692 ts_tablet_manager.cc:1367] T 4423a78b96cd42f8a509dc1f28d87a7f P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.066s user 0.002s sys 0.000s
I20240502 07:59:02.950286 27692 raft_consensus.cc:348] T 4423a78b96cd42f8a509dc1f28d87a7f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.950438 27692 raft_consensus.cc:374] T 4423a78b96cd42f8a509dc1f28d87a7f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.950474 27692 raft_consensus.cc:729] T 4423a78b96cd42f8a509dc1f28d87a7f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.950627 27692 consensus_queue.cc:260] T 4423a78b96cd42f8a509dc1f28d87a7f P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.950923 27692 ts_tablet_manager.cc:1398] T 4423a78b96cd42f8a509dc1f28d87a7f P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:02.952345 27693 ts_tablet_manager.cc:1367] T 5d342c368796426c81bede7ddc23937b P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.068s user 0.002s sys 0.000s
I20240502 07:59:02.952814 27693 raft_consensus.cc:348] T 5d342c368796426c81bede7ddc23937b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.952910 27693 raft_consensus.cc:374] T 5d342c368796426c81bede7ddc23937b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.952937 27693 raft_consensus.cc:729] T 5d342c368796426c81bede7ddc23937b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.953053 27693 consensus_queue.cc:260] T 5d342c368796426c81bede7ddc23937b P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:02.953367 27693 ts_tablet_manager.cc:1398] T 5d342c368796426c81bede7ddc23937b P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:02.956903 27496 consensus_queue.cc:1035] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:02.957885 27496 raft_consensus.cc:482] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:02.957973 27496 raft_consensus.cc:504] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.958298 27496 leader_election.cc:288] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:02.958801 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "712a88cd545745e8937d295266e85f04" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:02.958930 25083 raft_consensus.cc:2454] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:02.959417 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "712a88cd545745e8937d295266e85f04" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:02.959538 25214 raft_consensus.cc:2454] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:02.959558 22230 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:02.960031 24757 leader_election.cc:302] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:02.960199 27496 raft_consensus.cc:2781] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:02.960247 27496 raft_consensus.cc:482] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:02.960295 27496 raft_consensus.cc:3037] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:02.961309 27496 raft_consensus.cc:504] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.962410 27496 leader_election.cc:288] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:02.963128 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "712a88cd545745e8937d295266e85f04" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:02.963238 25214 raft_consensus.cc:3037] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:02.963253 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "712a88cd545745e8937d295266e85f04" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:02.963362 25083 raft_consensus.cc:3037] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:02.964210 25214 raft_consensus.cc:2454] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:02.964335 25083 raft_consensus.cc:2454] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:02.964721 24757 leader_election.cc:302] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:02.965410 27496 raft_consensus.cc:2781] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:02.965490 27496 raft_consensus.cc:686] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:59:02.965626 27496 consensus_queue.cc:237] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:02.967339 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 35847d860eb144fb9be1cdf6ee62c108 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2452e972d12a44fab560a9b62571021b]), partition=
I20240502 07:59:02.967440 25195 tablet_service.cc:1467] Processing CreateTablet for tablet 35847d860eb144fb9be1cdf6ee62c108 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2452e972d12a44fab560a9b62571021b]), partition=
I20240502 07:59:02.967523 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 35847d860eb144fb9be1cdf6ee62c108 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2452e972d12a44fab560a9b62571021b]), partition=
I20240502 07:59:02.967589 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35847d860eb144fb9be1cdf6ee62c108. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.967679 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35847d860eb144fb9be1cdf6ee62c108. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.967720 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35847d860eb144fb9be1cdf6ee62c108. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:02.970224 27695 tablet_bootstrap.cc:492] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:02.971158 27695 tablet_bootstrap.cc:654] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.972466 27693 tablet_bootstrap.cc:492] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:02.973023 22222 catalog_manager.cc:5484] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:02.976963 27695 tablet_bootstrap.cc:492] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:02.977080 27695 ts_tablet_manager.cc:1367] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:02.977422 27709 tablet_bootstrap.cc:492] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:02.977607 27695 raft_consensus.cc:348] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:02.977734 27695 raft_consensus.cc:374] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.977774 27695 raft_consensus.cc:729] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.977895 27695 consensus_queue.cc:260] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:02.978135 27695 ts_tablet_manager.cc:1398] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:02.979838 27693 tablet_bootstrap.cc:654] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.980314 27709 tablet_bootstrap.cc:654] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:02.980976 27693 tablet_bootstrap.cc:492] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:02.981103 27693 ts_tablet_manager.cc:1367] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20240502 07:59:02.981566 27693 raft_consensus.cc:348] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:02.981679 27693 raft_consensus.cc:374] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.981736 27693 raft_consensus.cc:729] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.981859 27693 consensus_queue.cc:260] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:02.982195 27693 ts_tablet_manager.cc:1398] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:02.984305 27709 tablet_bootstrap.cc:492] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:02.984836 27709 ts_tablet_manager.cc:1367] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:59:02.985428 27709 raft_consensus.cc:348] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:02.985605 27709 raft_consensus.cc:374] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:02.985653 27709 raft_consensus.cc:729] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:02.985770 27709 consensus_queue.cc:260] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:02.986619 27709 ts_tablet_manager.cc:1398] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:03.006460 27323 raft_consensus.cc:482] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:03.006593 27323 raft_consensus.cc:504] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:03.006654 24952 consensus_queue.cc:237] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:03.007081 27323 leader_election.cc:288] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:03.007491 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4423a78b96cd42f8a509dc1f28d87a7f" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:03.007638 24820 raft_consensus.cc:2454] T 4423a78b96cd42f8a509dc1f28d87a7f P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:03.008179 24820 raft_consensus.cc:1261] T cba71d2a28984e84b1950d59ec2e5a6a P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:03.008535 22763 raft_consensus.cc:1261] T cba71d2a28984e84b1950d59ec2e5a6a P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:03.008589 26758 consensus_queue.cc:1035] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:03.008889 25019 leader_election.cc:302] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:03.009064 27323 raft_consensus.cc:2781] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:03.009121 27323 raft_consensus.cc:482] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:03.009166 27323 raft_consensus.cc:3037] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.010269 27323 raft_consensus.cc:504] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:03.010335 27254 raft_consensus.cc:2932] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:03.010663 27323 leader_election.cc:288] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:03.010776 24820 raft_consensus.cc:2932] T cba71d2a28984e84b1950d59ec2e5a6a P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:03.010900 26758 consensus_queue.cc:1035] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:03.011076 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4423a78b96cd42f8a509dc1f28d87a7f" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:03.011186 22764 raft_consensus.cc:2454] T 4423a78b96cd42f8a509dc1f28d87a7f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:03.012097 22764 raft_consensus.cc:2932] T cba71d2a28984e84b1950d59ec2e5a6a P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:03.012326 27323 raft_consensus.cc:482] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:03.012405 27323 raft_consensus.cc:504] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:03.012701 27323 leader_election.cc:288] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:03.013142 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d342c368796426c81bede7ddc23937b" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:03.013249 24952 raft_consensus.cc:2454] T 5d342c368796426c81bede7ddc23937b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:03.013305 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4423a78b96cd42f8a509dc1f28d87a7f" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:03.013427 22763 raft_consensus.cc:3037] T 4423a78b96cd42f8a509dc1f28d87a7f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.013363 22222 catalog_manager.cc:5484] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 reported cstate change: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:03.013792 25020 leader_election.cc:302] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
I20240502 07:59:03.013947 27323 raft_consensus.cc:2781] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:03.013995 27323 raft_consensus.cc:482] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:03.014026 27323 raft_consensus.cc:3037] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.014250 22763 raft_consensus.cc:2454] T 4423a78b96cd42f8a509dc1f28d87a7f P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:03.014603 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d342c368796426c81bede7ddc23937b" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:03.014742 25214 raft_consensus.cc:2454] T 5d342c368796426c81bede7ddc23937b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:03.014802 27323 raft_consensus.cc:504] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:03.015313 25020 leader_election.cc:302] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:03.015512 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4423a78b96cd42f8a509dc1f28d87a7f" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:03.015606 24820 raft_consensus.cc:3037] T 4423a78b96cd42f8a509dc1f28d87a7f P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.016196 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d342c368796426c81bede7ddc23937b" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:03.016297 24952 raft_consensus.cc:3037] T 5d342c368796426c81bede7ddc23937b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.016407 24820 raft_consensus.cc:2454] T 4423a78b96cd42f8a509dc1f28d87a7f P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:03.016491 27658 raft_consensus.cc:2781] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:03.016640 27658 raft_consensus.cc:686] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:03.016763 27658 consensus_queue.cc:237] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:03.017252 24952 raft_consensus.cc:2454] T 5d342c368796426c81bede7ddc23937b P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:03.017836 25020 leader_election.cc:302] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
I20240502 07:59:03.019085 22228 catalog_manager.cc:5484] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:03.019510 25062 tablet_service.cc:1514] Processing DeleteTablet for tablet cba71d2a28984e84b1950d59ec2e5a6a with delete_type TABLET_DATA_TOMBSTONED (TS 227eec4b5f014d349661f6e0b39b9055 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:03.019651 27658 raft_consensus.cc:2781] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:03.019729 27658 raft_consensus.cc:686] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:03.019846 27658 consensus_queue.cc:237] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:03.021507 27323 leader_election.cc:288] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:03.021745 27733 tablet_replica.cc:330] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:03.021915 27733 raft_consensus.cc:2229] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:03.022053 27733 raft_consensus.cc:2258] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:03.022461 27733 ts_tablet_manager.cc:1875] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:03.022505 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d342c368796426c81bede7ddc23937b" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:03.022660 25214 raft_consensus.cc:3037] T 5d342c368796426c81bede7ddc23937b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.023695 25214 raft_consensus.cc:2454] T 5d342c368796426c81bede7ddc23937b P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:03.024422 22228 catalog_manager.cc:5484] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:03.024646 27733 ts_tablet_manager.cc:1888] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:03.024696 27733 log.cc:1196] T cba71d2a28984e84b1950d59ec2e5a6a P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/cba71d2a28984e84b1950d59ec2e5a6a
I20240502 07:59:03.029971 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet cba71d2a28984e84b1950d59ec2e5a6a (table rebalancer_extra_table_10 [id=03e205afe2d642629a8ba4c48516b9d5]) successfully deleted
I20240502 07:59:03.052300 27363 raft_consensus.cc:482] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:03.052448 27363 raft_consensus.cc:504] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:03.052877 27363 leader_election.cc:288] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:03.053470 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35847d860eb144fb9be1cdf6ee62c108" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:03.053625 24952 raft_consensus.cc:2454] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:03.053920 25151 leader_election.cc:302] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:03.055207 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35847d860eb144fb9be1cdf6ee62c108" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:03.055397 22371 raft_consensus.cc:2454] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:03.055768 27363 raft_consensus.cc:2781] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:03.055835 27363 raft_consensus.cc:482] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:03.055877 27363 raft_consensus.cc:3037] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.056902 27363 raft_consensus.cc:504] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:03.057271 27363 leader_election.cc:288] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:03.057849 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35847d860eb144fb9be1cdf6ee62c108" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:03.057977 24952 raft_consensus.cc:3037] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.058888 24952 raft_consensus.cc:2454] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:03.059286 25151 leader_election.cc:302] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:03.060371 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35847d860eb144fb9be1cdf6ee62c108" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:03.060493 22371 raft_consensus.cc:3037] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.061312 22371 raft_consensus.cc:2454] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:03.062335 27363 raft_consensus.cc:2781] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:03.062433 27363 raft_consensus.cc:686] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:03.062582 27363 consensus_queue.cc:237] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:03.065165 22228 catalog_manager.cc:5484] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:03.115680 25062 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:03.125640 22352 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:03.127523 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:03.131736 24932 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:03.133482 22612 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:03.139500 22483 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:03.146023 24799 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:03.155200 25195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:03.181121 22222 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:03.185247 24952 raft_consensus.cc:1261] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:03.185668 22371 raft_consensus.cc:1261] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:03.185689 27752 consensus_queue.cc:1035] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:03.186621 27752 consensus_queue.cc:1035] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:03.189466 22230 catalog_manager.cc:5576] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995) does not have the latest schema for tablet 35847d860eb144fb9be1cdf6ee62c108 (table rebalancer_dynamic_table [id=2452e972d12a44fab560a9b62571021b]). Expected version 1 got 0
I20240502 07:59:03.190157 22222 catalog_manager.cc:5576] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535) does not have the latest schema for tablet 35847d860eb144fb9be1cdf6ee62c108 (table rebalancer_dynamic_table [id=2452e972d12a44fab560a9b62571021b]). Expected version 1 got 0
I20240502 07:59:03.191277 27568 tablet.cc:1722] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:03.191601 27568 tablet.cc:1620] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:03.193833 27794 tablet.cc:1722] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:03.193862 27793 tablet.cc:1722] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:03.194188 27793 tablet.cc:1620] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:03.194459 27794 tablet.cc:1620] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:03.195004 22230 catalog_manager.cc:5576] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535) does not have the latest schema for tablet 35847d860eb144fb9be1cdf6ee62c108 (table rebalancer_dynamic_table [id=2452e972d12a44fab560a9b62571021b]). Expected version 1 got 0
I20240502 07:59:03.196199 22222 catalog_manager.cc:5576] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995) does not have the latest schema for tablet 35847d860eb144fb9be1cdf6ee62c108 (table rebalancer_dynamic_table [id=2452e972d12a44fab560a9b62571021b]). Expected version 1 got 0
I20240502 07:59:03.199793 22230 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=2452e972d12a44fab560a9b62571021b] alter complete (version 1)
I20240502 07:59:03.214325 22634 consensus_queue.cc:237] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.215497 22371 raft_consensus.cc:1261] T 41b4d8a50f684c19988a4cf1ee40921c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:03.215631 24952 raft_consensus.cc:1261] T 41b4d8a50f684c19988a4cf1ee40921c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:03.215863 27288 consensus_queue.cc:1035] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:03.216039 25587 consensus_queue.cc:1035] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:03.217768 25587 raft_consensus.cc:2932] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:03.218465 22371 raft_consensus.cc:2932] T 41b4d8a50f684c19988a4cf1ee40921c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:03.218717 24952 raft_consensus.cc:2932] T 41b4d8a50f684c19988a4cf1ee40921c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:03.223459 22230 catalog_manager.cc:5484] T 41b4d8a50f684c19988a4cf1ee40921c P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:03.227980 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet 41b4d8a50f684c19988a4cf1ee40921c with delete_type TABLET_DATA_TOMBSTONED (TS fabddb16c85944648a710ec52fc7244c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:03.228266 27680 tablet_replica.cc:330] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:03.228416 27680 raft_consensus.cc:2229] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:03.228543 27680 raft_consensus.cc:2258] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:03.228972 27680 ts_tablet_manager.cc:1875] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:03.230827 27680 ts_tablet_manager.cc:1888] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:03.230890 27680 log.cc:1196] T 41b4d8a50f684c19988a4cf1ee40921c P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/41b4d8a50f684c19988a4cf1ee40921c
I20240502 07:59:03.231451 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 41b4d8a50f684c19988a4cf1ee40921c (table rebalance_test_table_19 [id=4e87626760fa4d8098e55dfa6d1c55a9]) successfully deleted
I20240502 07:59:03.304991 22230 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:03.311141 27568 tablet.cc:1722] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:03.311489 27568 tablet.cc:1620] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:03.311915 27794 tablet.cc:1722] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:03.312196 27794 tablet.cc:1620] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:03.312320 27793 tablet.cc:1722] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:03.312589 27793 tablet.cc:1620] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:03.317206 22225 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=2452e972d12a44fab560a9b62571021b] alter complete (version 2)
I20240502 07:59:03.331848 24932 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:03.332669 25062 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:03.333506 25195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:03.337615 24799 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:03.338920 22352 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:03.348068 22612 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:03.361047 22483 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:03.369899 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:03.413038 26192 consensus_queue.cc:1035] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:03.416082 27496 consensus_queue.cc:1035] T 712a88cd545745e8937d295266e85f04 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:03.416378 22500 consensus_queue.cc:237] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:03.418200 22371 raft_consensus.cc:1261] T f49fcc586e6f498e93723d97b99c077e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:03.418609 27888 consensus_queue.cc:1035] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:03.418812 25084 raft_consensus.cc:1261] T f49fcc586e6f498e93723d97b99c077e P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:03.419293 27888 consensus_queue.cc:1035] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:03.419960 22230 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:03.423310 26965 raft_consensus.cc:2932] T f49fcc586e6f498e93723d97b99c077e P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:03.423717 22371 raft_consensus.cc:2932] T f49fcc586e6f498e93723d97b99c077e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:03.427006 22230 catalog_manager.cc:5484] T f49fcc586e6f498e93723d97b99c077e P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index 2 to 3, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) evicted. New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:03.427354 27568 tablet.cc:1722] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:03.431324 27794 tablet.cc:1722] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:03.435271 27793 tablet.cc:1722] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:03.436182 22226 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=2452e972d12a44fab560a9b62571021b] alter complete (version 3)
I20240502 07:59:03.436576 25195 tablet_service.cc:1514] Processing DeleteTablet for tablet f49fcc586e6f498e93723d97b99c077e with delete_type TABLET_DATA_TOMBSTONED (TS 43930dee0cd5498fa8e357a34dd71432 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:03.437835 27900 tablet_replica.cc:330] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:03.438030 27900 raft_consensus.cc:2229] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:03.438220 27900 raft_consensus.cc:2258] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:03.439563 25084 raft_consensus.cc:2932] T f49fcc586e6f498e93723d97b99c077e P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:03.442075 27900 ts_tablet_manager.cc:1875] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:03.445153 27900 ts_tablet_manager.cc:1888] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:03.445533 27900 log.cc:1196] T f49fcc586e6f498e93723d97b99c077e P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/f49fcc586e6f498e93723d97b99c077e
I20240502 07:59:03.446467 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet f49fcc586e6f498e93723d97b99c077e (table rebalancer_extra_table_10 [id=03e205afe2d642629a8ba4c48516b9d5]) successfully deleted
I20240502 07:59:03.451459 27731 consensus_queue.cc:1035] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:03.456334 27731 consensus_queue.cc:1035] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:03.473300 27731 consensus_queue.cc:1035] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:03.479094 27323 consensus_queue.cc:1035] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:03.527828 22634 consensus_queue.cc:237] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
W20240502 07:59:03.529567 22570 consensus_peers.cc:485] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): Couldn't send request to peer 1aecef81733c47b3b468504934357231. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c5841aadb46447e685af47b70d0feff4. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:03.529563 22500 raft_consensus.cc:1261] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:03.529914 22764 raft_consensus.cc:1261] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:03.530274 27288 consensus_queue.cc:1035] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:03.530447 25587 consensus_queue.cc:1035] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:03.532681 27288 raft_consensus.cc:2932] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:03.533841 22500 raft_consensus.cc:2932] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:03.535578 22226 catalog_manager.cc:5484] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:03.535851 22764 raft_consensus.cc:2932] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:03.543395 22225 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:03.543550 22225 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:03.545020 22230 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:03.546399 22225 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 35847d860eb144fb9be1cdf6ee62c108
I20240502 07:59:03.547087 22352 tablet_service.cc:1514] Processing DeleteTablet for tablet 35847d860eb144fb9be1cdf6ee62c108 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:03 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:03.547500 27971 tablet_replica.cc:330] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:03.547595 25195 tablet_service.cc:1514] Processing DeleteTablet for tablet 35847d860eb144fb9be1cdf6ee62c108 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:03 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:03.547652 27971 raft_consensus.cc:2229] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:03.547748 24932 tablet_service.cc:1514] Processing DeleteTablet for tablet 35847d860eb144fb9be1cdf6ee62c108 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:03 UTC) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:03.547791 27900 tablet_replica.cc:330] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:03.547925 27900 raft_consensus.cc:2229] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:03.548038 27972 tablet_replica.cc:330] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:03.548144 27900 raft_consensus.cc:2258] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:03.548173 27972 raft_consensus.cc:2229] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:03.548287 27972 raft_consensus.cc:2258] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:03.548635 27971 raft_consensus.cc:2258] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:03.548758 27900 ts_tablet_manager.cc:1875] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:03.549265 27971 ts_tablet_manager.cc:1875] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:03.551167 27900 ts_tablet_manager.cc:1888] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:03.551231 27900 log.cc:1196] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/35847d860eb144fb9be1cdf6ee62c108
I20240502 07:59:03.551481 27900 ts_tablet_manager.cc:1909] T 35847d860eb144fb9be1cdf6ee62c108 P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:59:03.552107 27972 ts_tablet_manager.cc:1875] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:03.552413 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 35847d860eb144fb9be1cdf6ee62c108 (table rebalancer_dynamic_table_new_name [id=2452e972d12a44fab560a9b62571021b]) successfully deleted
I20240502 07:59:03.553798 22612 tablet_service.cc:1467] Processing CreateTablet for tablet a0791c02721040e495462142c7bc9e35 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=7472e589acd942d3a925f8309b076ca6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:03.553893 27971 ts_tablet_manager.cc:1888] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:03.553972 27971 log.cc:1196] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/35847d860eb144fb9be1cdf6ee62c108
I20240502 07:59:03.553903 24799 tablet_service.cc:1467] Processing CreateTablet for tablet a0791c02721040e495462142c7bc9e35 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=7472e589acd942d3a925f8309b076ca6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:03.554088 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0791c02721040e495462142c7bc9e35. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.554212 24799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0791c02721040e495462142c7bc9e35. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.554299 27972 ts_tablet_manager.cc:1888] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:03.554325 27971 ts_tablet_manager.cc:1909] T 35847d860eb144fb9be1cdf6ee62c108 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:59:03.554369 27972 log.cc:1196] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/35847d860eb144fb9be1cdf6ee62c108
I20240502 07:59:03.554632 27972 ts_tablet_manager.cc:1909] T 35847d860eb144fb9be1cdf6ee62c108 P 1aecef81733c47b3b468504934357231: Deleting consensus metadata
I20240502 07:59:03.556269 25062 tablet_service.cc:1467] Processing CreateTablet for tablet 461098a3f8e64fe89a8603f78dc36dd2 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=7472e589acd942d3a925f8309b076ca6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:03.556576 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 461098a3f8e64fe89a8603f78dc36dd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.556563 27978 tablet_bootstrap.cc:492] T a0791c02721040e495462142c7bc9e35 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:03.557112 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 461098a3f8e64fe89a8603f78dc36dd2 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=7472e589acd942d3a925f8309b076ca6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:03.557300 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 461098a3f8e64fe89a8603f78dc36dd2 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=7472e589acd942d3a925f8309b076ca6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:03.557404 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 461098a3f8e64fe89a8603f78dc36dd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.557552 27978 tablet_bootstrap.cc:654] T a0791c02721040e495462142c7bc9e35 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:03.557612 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 461098a3f8e64fe89a8603f78dc36dd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.558782 27981 tablet_bootstrap.cc:492] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:03.558977 25063 tablet_service.cc:1467] Processing CreateTablet for tablet a0791c02721040e495462142c7bc9e35 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=7472e589acd942d3a925f8309b076ca6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:03.559187 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0791c02721040e495462142c7bc9e35. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.559839 27981 tablet_bootstrap.cc:654] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:03.561053 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet 35847d860eb144fb9be1cdf6ee62c108 (table rebalancer_dynamic_table_new_name [id=2452e972d12a44fab560a9b62571021b]) successfully deleted
I20240502 07:59:03.561285 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 35847d860eb144fb9be1cdf6ee62c108 (table rebalancer_dynamic_table_new_name [id=2452e972d12a44fab560a9b62571021b]) successfully deleted
I20240502 07:59:03.561558 27979 tablet_bootstrap.cc:492] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:03.561607 25063 tablet_service.cc:1467] Processing CreateTablet for tablet a5fc34a6c3384261842e2aab29088688 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=7472e589acd942d3a925f8309b076ca6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:03.561810 27981 tablet_bootstrap.cc:492] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:03.561851 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5fc34a6c3384261842e2aab29088688. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.561895 27981 ts_tablet_manager.cc:1367] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20240502 07:59:03.562151 24932 tablet_service.cc:1467] Processing CreateTablet for tablet a5fc34a6c3384261842e2aab29088688 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=7472e589acd942d3a925f8309b076ca6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:03.562391 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5fc34a6c3384261842e2aab29088688. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.562346 27981 raft_consensus.cc:348] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.562458 27981 raft_consensus.cc:374] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:03.562489 27981 raft_consensus.cc:729] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:03.562849 22483 tablet_service.cc:1467] Processing CreateTablet for tablet a5fc34a6c3384261842e2aab29088688 (DEFAULT_TABLE table=rebalancer_extra_table_18 [id=7472e589acd942d3a925f8309b076ca6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:03.562984 27981 consensus_queue.cc:260] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.563094 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5fc34a6c3384261842e2aab29088688. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.565546 27983 tablet_bootstrap.cc:492] T 461098a3f8e64fe89a8603f78dc36dd2 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:03.565821 27981 ts_tablet_manager.cc:1398] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:03.566349 27978 tablet_bootstrap.cc:492] T a0791c02721040e495462142c7bc9e35 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:03.566444 27978 ts_tablet_manager.cc:1367] T a0791c02721040e495462142c7bc9e35 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 07:59:03.566076 27979 tablet_bootstrap.cc:654] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:03.566709 27983 tablet_bootstrap.cc:654] T 461098a3f8e64fe89a8603f78dc36dd2 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:03.566973 27978 raft_consensus.cc:348] T a0791c02721040e495462142c7bc9e35 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:03.567073 27978 raft_consensus.cc:374] T a0791c02721040e495462142c7bc9e35 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:03.567111 27978 raft_consensus.cc:729] T a0791c02721040e495462142c7bc9e35 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:03.567256 27978 consensus_queue.cc:260] T a0791c02721040e495462142c7bc9e35 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:03.567552 27978 ts_tablet_manager.cc:1398] T a0791c02721040e495462142c7bc9e35 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:03.567833 27979 tablet_bootstrap.cc:492] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:03.567919 27979 ts_tablet_manager.cc:1367] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:03.567992 27981 tablet_bootstrap.cc:492] T a0791c02721040e495462142c7bc9e35 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:03.568302 27979 raft_consensus.cc:348] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:03.568439 27979 raft_consensus.cc:374] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:03.568476 27979 raft_consensus.cc:729] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:03.568595 27979 consensus_queue.cc:260] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:03.568889 27979 ts_tablet_manager.cc:1398] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:03.567378 27982 tablet_bootstrap.cc:492] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:03.571348 27981 tablet_bootstrap.cc:654] T a0791c02721040e495462142c7bc9e35 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:03.571722 27982 tablet_bootstrap.cc:654] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:03.573101 27981 tablet_bootstrap.cc:492] T a0791c02721040e495462142c7bc9e35 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:03.573150 27983 tablet_bootstrap.cc:492] T 461098a3f8e64fe89a8603f78dc36dd2 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:03.573197 27981 ts_tablet_manager.cc:1367] T a0791c02721040e495462142c7bc9e35 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:03.573249 27983 ts_tablet_manager.cc:1367] T 461098a3f8e64fe89a8603f78dc36dd2 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:59:03.573282 27982 tablet_bootstrap.cc:492] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:03.573408 27982 ts_tablet_manager.cc:1367] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:03.573736 27981 raft_consensus.cc:348] T a0791c02721040e495462142c7bc9e35 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:03.573848 27981 raft_consensus.cc:374] T a0791c02721040e495462142c7bc9e35 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:03.573882 27981 raft_consensus.cc:729] T a0791c02721040e495462142c7bc9e35 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:03.574016 27981 consensus_queue.cc:260] T a0791c02721040e495462142c7bc9e35 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:03.574398 27981 ts_tablet_manager.cc:1398] T a0791c02721040e495462142c7bc9e35 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:03.574591 27981 tablet_bootstrap.cc:492] T a5fc34a6c3384261842e2aab29088688 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:03.574513 27983 raft_consensus.cc:348] T 461098a3f8e64fe89a8603f78dc36dd2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.574635 27983 raft_consensus.cc:374] T 461098a3f8e64fe89a8603f78dc36dd2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:03.574679 27983 raft_consensus.cc:729] T 461098a3f8e64fe89a8603f78dc36dd2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:03.574802 27983 consensus_queue.cc:260] T 461098a3f8e64fe89a8603f78dc36dd2 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.573882 27982 raft_consensus.cc:348] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.575297 27982 raft_consensus.cc:374] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:03.575353 27982 raft_consensus.cc:729] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:03.575477 27982 consensus_queue.cc:260] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.575686 27981 tablet_bootstrap.cc:654] T a5fc34a6c3384261842e2aab29088688 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:03.577010 27983 ts_tablet_manager.cc:1398] T 461098a3f8e64fe89a8603f78dc36dd2 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:03.577245 27983 tablet_bootstrap.cc:492] T a5fc34a6c3384261842e2aab29088688 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:03.578377 27983 tablet_bootstrap.cc:654] T a5fc34a6c3384261842e2aab29088688 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:03.580264 27982 ts_tablet_manager.cc:1398] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240502 07:59:03.580376 27983 tablet_bootstrap.cc:492] T a5fc34a6c3384261842e2aab29088688 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:03.580482 27983 ts_tablet_manager.cc:1367] T a5fc34a6c3384261842e2aab29088688 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:03.580920 27983 raft_consensus.cc:348] T a5fc34a6c3384261842e2aab29088688 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.581024 27983 raft_consensus.cc:374] T a5fc34a6c3384261842e2aab29088688 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:03.581054 27983 raft_consensus.cc:729] T a5fc34a6c3384261842e2aab29088688 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:03.581194 27983 consensus_queue.cc:260] T a5fc34a6c3384261842e2aab29088688 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.581534 27983 ts_tablet_manager.cc:1398] T a5fc34a6c3384261842e2aab29088688 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:03.581944 27982 tablet_bootstrap.cc:492] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:03.583319 27982 tablet_bootstrap.cc:654] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:03.584937 27982 tablet_bootstrap.cc:492] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:03.585026 27982 ts_tablet_manager.cc:1367] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:03.585119 27981 tablet_bootstrap.cc:492] T a5fc34a6c3384261842e2aab29088688 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:03.585214 27981 ts_tablet_manager.cc:1367] T a5fc34a6c3384261842e2aab29088688 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 07:59:03.585772 27981 raft_consensus.cc:348] T a5fc34a6c3384261842e2aab29088688 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.585796 27982 raft_consensus.cc:348] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.585899 27982 raft_consensus.cc:374] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:03.585932 27982 raft_consensus.cc:729] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:03.585893 27981 raft_consensus.cc:374] T a5fc34a6c3384261842e2aab29088688 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:03.586061 27981 raft_consensus.cc:729] T a5fc34a6c3384261842e2aab29088688 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:03.586048 27982 consensus_queue.cc:260] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.586195 27981 consensus_queue.cc:260] T a5fc34a6c3384261842e2aab29088688 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.588930 27982 ts_tablet_manager.cc:1398] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:03.589452 27981 ts_tablet_manager.cc:1398] T a5fc34a6c3384261842e2aab29088688 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240502 07:59:03.592932 25587 raft_consensus.cc:482] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:03.593029 25587 raft_consensus.cc:504] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:03.593623 25587 leader_election.cc:288] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:03.593938 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0791c02721040e495462142c7bc9e35" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:03.594091 24820 raft_consensus.cc:2454] T a0791c02721040e495462142c7bc9e35 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:03.594110 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0791c02721040e495462142c7bc9e35" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:03.594220 25084 raft_consensus.cc:2454] T a0791c02721040e495462142c7bc9e35 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:03.594575 22569 leader_election.cc:302] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:03.594877 25587 raft_consensus.cc:2781] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:03.594942 25587 raft_consensus.cc:482] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:03.594981 25587 raft_consensus.cc:3037] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.596649 25587 raft_consensus.cc:504] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:03.596997 25587 leader_election.cc:288] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:03.597360 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0791c02721040e495462142c7bc9e35" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:03.597476 25084 raft_consensus.cc:3037] T a0791c02721040e495462142c7bc9e35 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.598289 25084 raft_consensus.cc:2454] T a0791c02721040e495462142c7bc9e35 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:03.598506 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0791c02721040e495462142c7bc9e35" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:03.598553 22567 leader_election.cc:302] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:03.598625 24820 raft_consensus.cc:3037] T a0791c02721040e495462142c7bc9e35 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.598759 25587 raft_consensus.cc:2781] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:03.598837 25587 raft_consensus.cc:686] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:59:03.598969 25587 consensus_queue.cc:237] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:03.599246 22634 consensus_queue.cc:237] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.599454 24820 raft_consensus.cc:2454] T a0791c02721040e495462142c7bc9e35 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:03.600879 22226 catalog_manager.cc:5484] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:03.602135 22372 raft_consensus.cc:1261] T b7cdcee32c8f4cd3af30c402b8da2484 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:03.602301 22500 raft_consensus.cc:1261] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:03.602361 22570 consensus_peers.cc:485] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): Couldn't send request to peer 1aecef81733c47b3b468504934357231. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7cdcee32c8f4cd3af30c402b8da2484. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:03.602669 25587 consensus_queue.cc:1035] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:03.602814 27965 consensus_queue.cc:1035] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:03.606380 25587 raft_consensus.cc:2932] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:03.608670 22500 raft_consensus.cc:2932] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:03.608829 22372 raft_consensus.cc:2932] T b7cdcee32c8f4cd3af30c402b8da2484 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:03.614168 22226 catalog_manager.cc:5484] T b7cdcee32c8f4cd3af30c402b8da2484 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index -1 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:03.617909 26758 raft_consensus.cc:482] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:03.618011 26758 raft_consensus.cc:504] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.618422 26758 leader_election.cc:288] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:03.619458 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461098a3f8e64fe89a8603f78dc36dd2" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:03.619582 25084 raft_consensus.cc:2454] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:03.620390 24885 leader_election.cc:302] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
I20240502 07:59:03.621871 26758 raft_consensus.cc:2781] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:03.621927 26758 raft_consensus.cc:482] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:03.621959 26758 raft_consensus.cc:3037] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.622918 26758 raft_consensus.cc:504] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.623239 26758 leader_election.cc:288] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:03.623663 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461098a3f8e64fe89a8603f78dc36dd2" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:03.623775 25084 raft_consensus.cc:3037] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.624682 25084 raft_consensus.cc:2454] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:03.624995 24885 leader_election.cc:302] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
I20240502 07:59:03.625156 26758 raft_consensus.cc:2781] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:03.625342 26758 raft_consensus.cc:686] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:03.625474 26758 consensus_queue.cc:237] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.627715 22226 catalog_manager.cc:5484] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:03.635390 22500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461098a3f8e64fe89a8603f78dc36dd2" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:03.635592 22500 raft_consensus.cc:2454] T 461098a3f8e64fe89a8603f78dc36dd2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:03.635860 22499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461098a3f8e64fe89a8603f78dc36dd2" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:03.635972 22499 raft_consensus.cc:3037] T 461098a3f8e64fe89a8603f78dc36dd2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.637207 22499 raft_consensus.cc:2454] T 461098a3f8e64fe89a8603f78dc36dd2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:03.648188 22226 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:03.650730 22499 consensus_queue.cc:237] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:03.652833 22372 raft_consensus.cc:1261] T dbf60f171dfe4c88840587bf0df64cac P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:03.653215 22634 raft_consensus.cc:1261] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:03.653788 27893 consensus_queue.cc:1035] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:03.654165 26965 consensus_queue.cc:1035] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:03.654364 22439 consensus_peers.cc:485] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c -> Peer 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): Couldn't send request to peer 43930dee0cd5498fa8e357a34dd71432. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dbf60f171dfe4c88840587bf0df64cac. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:03.654461 25195 tablet_service.cc:1467] Processing CreateTablet for tablet 7eaa8fb0cbc2467a9f512e6892bdb777 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=fb52db55839b4ca6b11a412261f5d725]), partition=
I20240502 07:59:03.654477 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 7eaa8fb0cbc2467a9f512e6892bdb777 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=fb52db55839b4ca6b11a412261f5d725]), partition=
I20240502 07:59:03.654727 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eaa8fb0cbc2467a9f512e6892bdb777. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.654737 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eaa8fb0cbc2467a9f512e6892bdb777. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.656812 27982 tablet_bootstrap.cc:492] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:03.657109 28024 tablet_bootstrap.cc:492] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:03.657306 27893 raft_consensus.cc:2932] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:03.657773 27982 tablet_bootstrap.cc:654] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:03.658164 28024 tablet_bootstrap.cc:654] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:03.659298 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 7eaa8fb0cbc2467a9f512e6892bdb777 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=fb52db55839b4ca6b11a412261f5d725]), partition=
I20240502 07:59:03.659575 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eaa8fb0cbc2467a9f512e6892bdb777. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:03.659678 22372 raft_consensus.cc:2932] T dbf60f171dfe4c88840587bf0df64cac P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:03.660048 28024 tablet_bootstrap.cc:492] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:03.660110 28024 ts_tablet_manager.cc:1367] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:03.660439 28024 raft_consensus.cc:348] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:03.660507 28024 raft_consensus.cc:374] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:03.660527 28024 raft_consensus.cc:729] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:03.660636 28024 consensus_queue.cc:260] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:03.660984 28024 ts_tablet_manager.cc:1398] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:03.661762 22634 raft_consensus.cc:2932] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:03.662401 22226 catalog_manager.cc:5484] T dbf60f171dfe4c88840587bf0df64cac P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:03.664602 27982 tablet_bootstrap.cc:492] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:03.664695 27982 ts_tablet_manager.cc:1367] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:59:03.665165 27982 raft_consensus.cc:348] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:03.665279 27982 raft_consensus.cc:374] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:03.665314 27982 raft_consensus.cc:729] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:03.665464 27982 consensus_queue.cc:260] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:03.665774 27982 ts_tablet_manager.cc:1398] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:03.667613 26758 raft_consensus.cc:482] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:03.667711 26758 raft_consensus.cc:504] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.668051 26758 leader_election.cc:288] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:03.668522 22499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5fc34a6c3384261842e2aab29088688" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:03.668545 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5fc34a6c3384261842e2aab29088688" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:03.668661 22499 raft_consensus.cc:2454] T a5fc34a6c3384261842e2aab29088688 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:03.668687 25084 raft_consensus.cc:2454] T a5fc34a6c3384261842e2aab29088688 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:03.669922 28028 tablet_bootstrap.cc:492] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:03.670620 24885 leader_election.cc:302] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:03.670918 26758 raft_consensus.cc:2781] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:03.670977 26758 raft_consensus.cc:482] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:03.671033 26758 raft_consensus.cc:3037] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.671109 28028 tablet_bootstrap.cc:654] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:03.672101 26758 raft_consensus.cc:504] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.672363 28028 tablet_bootstrap.cc:492] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:03.672451 28028 ts_tablet_manager.cc:1367] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:03.672461 26758 leader_election.cc:288] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:03.672850 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5fc34a6c3384261842e2aab29088688" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:03.672961 22499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5fc34a6c3384261842e2aab29088688" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:03.672972 25084 raft_consensus.cc:3037] T a5fc34a6c3384261842e2aab29088688 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.672946 28028 raft_consensus.cc:348] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:03.673065 28028 raft_consensus.cc:374] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:03.673065 22499 raft_consensus.cc:3037] T a5fc34a6c3384261842e2aab29088688 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.673100 28028 raft_consensus.cc:729] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:03.673243 28028 consensus_queue.cc:260] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:03.673544 28028 ts_tablet_manager.cc:1398] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:03.674080 22499 raft_consensus.cc:2454] T a5fc34a6c3384261842e2aab29088688 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:03.674360 25084 raft_consensus.cc:2454] T a5fc34a6c3384261842e2aab29088688 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:03.674463 24885 leader_election.cc:302] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:03.674782 26758 raft_consensus.cc:2781] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:03.674870 26758 raft_consensus.cc:686] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:03.675094 26758 consensus_queue.cc:237] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:03.676824 22230 catalog_manager.cc:5484] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:03.694193 22634 consensus_queue.cc:237] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:03.695658 22499 raft_consensus.cc:1261] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:03.695911 22569 consensus_peers.cc:485] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b270d89546214f4d9d15193f4f8e04d5. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:03.696027 27965 consensus_queue.cc:1035] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:03.696117 22764 raft_consensus.cc:1261] T b270d89546214f4d9d15193f4f8e04d5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:03.696604 27965 consensus_queue.cc:1035] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:03.698796 27288 raft_consensus.cc:2932] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:03.699101 22764 raft_consensus.cc:2932] T b270d89546214f4d9d15193f4f8e04d5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:03.700915 22499 raft_consensus.cc:2932] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:03.702204 22230 catalog_manager.cc:5484] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:03.722088 25214 consensus_queue.cc:237] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:03.723770 25084 raft_consensus.cc:1261] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:03.723837 24820 raft_consensus.cc:1261] T e3fd481f3c78497eb629b0180c07674f P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:03.724314 27752 consensus_queue.cc:1035] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 07:59:03.724539 25151 consensus_peers.cc:485] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 -> Peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): Couldn't send request to peer e51b10af49484aaa8f0267aebc1bf27f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e3fd481f3c78497eb629b0180c07674f. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:03.724754 27752 consensus_queue.cc:1035] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:03.726624 27751 raft_consensus.cc:2932] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:03.728740 24820 raft_consensus.cc:2932] T e3fd481f3c78497eb629b0180c07674f P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:03.729281 25084 raft_consensus.cc:2932] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:03.730875 22225 catalog_manager.cc:5484] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:03.755204 25084 consensus_queue.cc:237] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:03.756891 22372 raft_consensus.cc:1261] T f7828cb273c64934853b2cd38368402f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:03.756965 24952 raft_consensus.cc:1261] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:03.757447 28083 consensus_queue.cc:1035] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 07:59:03.757648 25020 consensus_peers.cc:485] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 -> Peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): Couldn't send request to peer e51b10af49484aaa8f0267aebc1bf27f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f7828cb273c64934853b2cd38368402f. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:03.757835 28083 consensus_queue.cc:1035] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:03.759950 27323 raft_consensus.cc:2932] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:03.761282 22372 raft_consensus.cc:2932] T f7828cb273c64934853b2cd38368402f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:03.764575 22229 catalog_manager.cc:5484] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:03.764590 24952 raft_consensus.cc:2932] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:03.787765 25214 consensus_queue.cc:237] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:03.789613 24952 raft_consensus.cc:1261] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:03.789620 22763 raft_consensus.cc:1261] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:03.789810 25148 consensus_peers.cc:485] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 -> Peer fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): Couldn't send request to peer fabddb16c85944648a710ec52fc7244c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: af4d275b38ab49e7ac63e2fb9ffd8c07. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:03.790422 27751 consensus_queue.cc:1035] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:03.790649 27363 consensus_queue.cc:1035] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:03.793219 27751 raft_consensus.cc:2932] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:03.793653 24952 raft_consensus.cc:2932] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:03.795861 22763 raft_consensus.cc:2932] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:03.796324 22230 catalog_manager.cc:5484] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:03.816452 22499 consensus_queue.cc:237] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:03.818012 25214 raft_consensus.cc:1261] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:03.818025 22372 raft_consensus.cc:1261] T cc97164c8cfe4f91afabcc606d9821ef P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:03.818070 22439 consensus_peers.cc:485] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c -> Peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): Couldn't send request to peer e51b10af49484aaa8f0267aebc1bf27f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc97164c8cfe4f91afabcc606d9821ef. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:03.818408 28026 consensus_queue.cc:1035] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:03.818621 27888 consensus_queue.cc:1035] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:03.820382 27888 raft_consensus.cc:2932] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:03.820807 22372 raft_consensus.cc:2932] T cc97164c8cfe4f91afabcc606d9821ef P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:03.823539 25214 raft_consensus.cc:2932] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:03.824023 22230 catalog_manager.cc:5484] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c reported cstate change: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:03.845500 25084 consensus_queue.cc:237] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:03.846985 22372 raft_consensus.cc:1261] T 9f44c905bfaa4db093a1b39b6c21e03c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:03.846985 24952 raft_consensus.cc:1261] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:03.847105 25019 consensus_peers.cc:485] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9f44c905bfaa4db093a1b39b6c21e03c. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:03.847412 27731 consensus_queue.cc:1035] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:03.847610 28086 consensus_queue.cc:1035] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:03.849996 28086 raft_consensus.cc:2932] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:03.850419 22372 raft_consensus.cc:2932] T 9f44c905bfaa4db093a1b39b6c21e03c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:03.851971 24952 raft_consensus.cc:2932] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:03.854409 22225 catalog_manager.cc:5484] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: config changed from index -1 to 2, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:03.875350 24952 consensus_queue.cc:237] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:03.877140 22372 raft_consensus.cc:1261] T 288b33b7a5f84af59bde5fae34cbe924 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:03.877158 25084 raft_consensus.cc:1261] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:03.877652 28159 consensus_queue.cc:1035] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 07:59:03.877882 24885 consensus_peers.cc:485] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 -> Peer fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): Couldn't send request to peer fabddb16c85944648a710ec52fc7244c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 288b33b7a5f84af59bde5fae34cbe924. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:03.878173 28159 consensus_queue.cc:1035] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:03.880862 28159 raft_consensus.cc:2932] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:03.881250 25084 raft_consensus.cc:2932] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:03.883275 22372 raft_consensus.cc:2932] T 288b33b7a5f84af59bde5fae34cbe924 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:03.884778 22230 catalog_manager.cc:5484] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:03.961617 28162 raft_consensus.cc:482] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:03.961771 28162 raft_consensus.cc:504] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:03.962265 28162 leader_election.cc:288] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:03.963443 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eaa8fb0cbc2467a9f512e6892bdb777" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:03.963627 25214 raft_consensus.cc:2454] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:03.963894 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eaa8fb0cbc2467a9f512e6892bdb777" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:03.964026 22372 raft_consensus.cc:2454] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:03.964571 24888 leader_election.cc:302] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:03.965195 28162 raft_consensus.cc:2781] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:03.965263 28162 raft_consensus.cc:482] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:03.965308 28162 raft_consensus.cc:3037] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.966307 28162 raft_consensus.cc:504] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:03.966673 28162 leader_election.cc:288] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:03.967108 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eaa8fb0cbc2467a9f512e6892bdb777" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:03.967175 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eaa8fb0cbc2467a9f512e6892bdb777" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:03.967226 25214 raft_consensus.cc:3037] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.967306 22371 raft_consensus.cc:3037] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:03.968279 22371 raft_consensus.cc:2454] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:03.968386 25214 raft_consensus.cc:2454] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:03.968876 24888 leader_election.cc:302] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:03.969069 28162 raft_consensus.cc:2781] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:03.969169 28162 raft_consensus.cc:686] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:03.969378 28162 consensus_queue.cc:237] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:03.971603 22230 catalog_manager.cc:5484] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:03.992096 22352 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:03.992123 24932 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:03.992859 22483 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:03.994349 25063 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:03.996274 27813 consensus_queue.cc:1035] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:04.001521 22612 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:04.001925 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:04.006258 27813 consensus_queue.cc:1035] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:04.016467 25195 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:04.032739 24799 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:04.058482 28162 consensus_queue.cc:1035] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:04.062935 28162 consensus_queue.cc:1035] T a5fc34a6c3384261842e2aab29088688 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:04.105906 28267 ts_tablet_manager.cc:902] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:04.106456 28267 tablet_copy_client.cc:308] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:59:04.107659 22653 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet c5841aadb46447e685af47b70d0feff4 from peer 1aecef81733c47b3b468504934357231 ({username='slave'} at 127.15.129.134:44071)
I20240502 07:59:04.107790 22653 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet c5841aadb46447e685af47b70d0feff4 from peer 1aecef81733c47b3b468504934357231 at {username='slave'} at 127.15.129.134:44071: session id = 1aecef81733c47b3b468504934357231-c5841aadb46447e685af47b70d0feff4
I20240502 07:59:04.108918 22653 tablet_copy_source_session.cc:205] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:04.112968 28267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5841aadb46447e685af47b70d0feff4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.115860 28267 tablet_copy_client.cc:783] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:04.116132 28267 tablet_copy_client.cc:647] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:04.118072 28267 tablet_copy_client.cc:523] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.119551 28267 tablet_bootstrap.cc:492] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:04.129732 28273 ts_tablet_manager.cc:902] T b270d89546214f4d9d15193f4f8e04d5 P 4b4e7eeb7f3f42119c0363a79febc54b: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:04.130425 28273 tablet_copy_client.cc:308] T b270d89546214f4d9d15193f4f8e04d5 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:59:04.131042 22653 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet b270d89546214f4d9d15193f4f8e04d5 from peer 4b4e7eeb7f3f42119c0363a79febc54b ({username='slave'} at 127.15.129.133:55755)
I20240502 07:59:04.131157 22653 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet b270d89546214f4d9d15193f4f8e04d5 from peer 4b4e7eeb7f3f42119c0363a79febc54b at {username='slave'} at 127.15.129.133:55755: session id = 4b4e7eeb7f3f42119c0363a79febc54b-b270d89546214f4d9d15193f4f8e04d5
I20240502 07:59:04.132354 22653 tablet_copy_source_session.cc:205] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:04.133939 28273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b270d89546214f4d9d15193f4f8e04d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.136665 28273 tablet_copy_client.cc:783] T b270d89546214f4d9d15193f4f8e04d5 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:04.136854 28273 tablet_copy_client.cc:647] T b270d89546214f4d9d15193f4f8e04d5 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:04.142449 22226 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:04.143782 28273 tablet_copy_client.cc:523] T b270d89546214f4d9d15193f4f8e04d5 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.145108 28273 tablet_bootstrap.cc:492] T b270d89546214f4d9d15193f4f8e04d5 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:04.149847 25215 raft_consensus.cc:1261] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:04.150830 28008 consensus_queue.cc:1035] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:04.151538 22371 raft_consensus.cc:1261] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:04.152426 28008 consensus_queue.cc:1035] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:04.153782 28267 tablet_bootstrap.cc:492] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.154160 28267 tablet_bootstrap.cc:492] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231: Bootstrap complete.
I20240502 07:59:04.154412 28267 ts_tablet_manager.cc:1367] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.035s user 0.024s sys 0.004s
I20240502 07:59:04.154946 28267 raft_consensus.cc:348] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:04.155051 28267 raft_consensus.cc:729] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.155201 28267 consensus_queue.cc:260] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:04.156126 28267 ts_tablet_manager.cc:1398] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:04.156910 22226 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 7eaa8fb0cbc2467a9f512e6892bdb777 (table rebalancer_dynamic_table [id=fb52db55839b4ca6b11a412261f5d725]). Expected version 1 got 0
I20240502 07:59:04.157523 22653 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session 1aecef81733c47b3b468504934357231-c5841aadb46447e685af47b70d0feff4 received from {username='slave'} at 127.15.129.134:44071
I20240502 07:59:04.157609 22653 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session 1aecef81733c47b3b468504934357231-c5841aadb46447e685af47b70d0feff4 on tablet c5841aadb46447e685af47b70d0feff4 with peer 1aecef81733c47b3b468504934357231
I20240502 07:59:04.160290 22226 catalog_manager.cc:5576] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535) does not have the latest schema for tablet 7eaa8fb0cbc2467a9f512e6892bdb777 (table rebalancer_dynamic_table [id=fb52db55839b4ca6b11a412261f5d725]). Expected version 1 got 0
I20240502 07:59:04.161087 28295 tablet.cc:1722] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:04.161181 28296 tablet.cc:1722] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:04.161360 25195 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:04.161470 28295 tablet.cc:1620] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:04.162381 28267 ts_tablet_manager.cc:902] T b7cdcee32c8f4cd3af30c402b8da2484 P 1aecef81733c47b3b468504934357231: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:04.163466 28267 tablet_copy_client.cc:308] T b7cdcee32c8f4cd3af30c402b8da2484 P 1aecef81733c47b3b468504934357231: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:59:04.163771 28296 tablet.cc:1620] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:04.164248 22653 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet b7cdcee32c8f4cd3af30c402b8da2484 from peer 1aecef81733c47b3b468504934357231 ({username='slave'} at 127.15.129.134:44071)
I20240502 07:59:04.164366 22653 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet b7cdcee32c8f4cd3af30c402b8da2484 from peer 1aecef81733c47b3b468504934357231 at {username='slave'} at 127.15.129.134:44071: session id = 1aecef81733c47b3b468504934357231-b7cdcee32c8f4cd3af30c402b8da2484
I20240502 07:59:04.165215 28301 tablet.cc:1722] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:04.165376 22653 tablet_copy_source_session.cc:205] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:04.165557 28301 tablet.cc:1620] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:04.166782 28267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7cdcee32c8f4cd3af30c402b8da2484. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.167083 22223 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 7eaa8fb0cbc2467a9f512e6892bdb777 (table rebalancer_dynamic_table [id=fb52db55839b4ca6b11a412261f5d725]). Expected version 1 got 0
I20240502 07:59:04.170513 28267 tablet_copy_client.cc:783] T b7cdcee32c8f4cd3af30c402b8da2484 P 1aecef81733c47b3b468504934357231: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:04.170656 28267 tablet_copy_client.cc:647] T b7cdcee32c8f4cd3af30c402b8da2484 P 1aecef81733c47b3b468504934357231: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:04.174357 28267 tablet_copy_client.cc:523] T b7cdcee32c8f4cd3af30c402b8da2484 P 1aecef81733c47b3b468504934357231: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.173976 22225 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=fb52db55839b4ca6b11a412261f5d725] alter complete (version 1)
I20240502 07:59:04.176219 28267 tablet_bootstrap.cc:492] T b7cdcee32c8f4cd3af30c402b8da2484 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:04.176416 24799 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:04.183668 22612 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:04.190343 28303 ts_tablet_manager.cc:902] T e3fd481f3c78497eb629b0180c07674f P e51b10af49484aaa8f0267aebc1bf27f: Initiating tablet copy from peer 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:04.192098 28303 tablet_copy_client.cc:308] T e3fd481f3c78497eb629b0180c07674f P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.136:36591
I20240502 07:59:04.193190 25235 tablet_copy_service.cc:140] P 43930dee0cd5498fa8e357a34dd71432: Received BeginTabletCopySession request for tablet e3fd481f3c78497eb629b0180c07674f from peer e51b10af49484aaa8f0267aebc1bf27f ({username='slave'} at 127.15.129.131:60465)
I20240502 07:59:04.193382 25235 tablet_copy_service.cc:161] P 43930dee0cd5498fa8e357a34dd71432: Beginning new tablet copy session on tablet e3fd481f3c78497eb629b0180c07674f from peer e51b10af49484aaa8f0267aebc1bf27f at {username='slave'} at 127.15.129.131:60465: session id = e51b10af49484aaa8f0267aebc1bf27f-e3fd481f3c78497eb629b0180c07674f
I20240502 07:59:04.194600 25235 tablet_copy_source_session.cc:205] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:04.195996 28303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3fd481f3c78497eb629b0180c07674f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.196302 22225 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_20"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:04.199254 28303 tablet_copy_client.cc:783] T e3fd481f3c78497eb629b0180c07674f P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:04.199476 28303 tablet_copy_client.cc:647] T e3fd481f3c78497eb629b0180c07674f P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:04.204480 24799 tablet_service.cc:1467] Processing CreateTablet for tablet 71d8cf6286aa4c3cb4f87d5974cf8548 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=3e579f694fc54bdba0aade1f5091d17b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:04.204809 24799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71d8cf6286aa4c3cb4f87d5974cf8548. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.205727 24932 tablet_service.cc:1467] Processing CreateTablet for tablet ecaf2063c4cb4d2dab58354270c60cb2 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=3e579f694fc54bdba0aade1f5091d17b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:04.205993 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecaf2063c4cb4d2dab58354270c60cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.206077 25194 tablet_service.cc:1467] Processing CreateTablet for tablet ecaf2063c4cb4d2dab58354270c60cb2 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=3e579f694fc54bdba0aade1f5091d17b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:04.206331 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecaf2063c4cb4d2dab58354270c60cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.206353 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 5b952c7cbc2447ab98c17bcb2f80c8ac (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=3e579f694fc54bdba0aade1f5091d17b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:04.206656 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b952c7cbc2447ab98c17bcb2f80c8ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.206745 22612 tablet_service.cc:1467] Processing CreateTablet for tablet 5b952c7cbc2447ab98c17bcb2f80c8ac (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=3e579f694fc54bdba0aade1f5091d17b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:04.207012 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b952c7cbc2447ab98c17bcb2f80c8ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.208550 22613 tablet_service.cc:1467] Processing CreateTablet for tablet ecaf2063c4cb4d2dab58354270c60cb2 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=3e579f694fc54bdba0aade1f5091d17b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:04.208537 25195 tablet_service.cc:1467] Processing CreateTablet for tablet 5b952c7cbc2447ab98c17bcb2f80c8ac (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=3e579f694fc54bdba0aade1f5091d17b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:04.208758 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecaf2063c4cb4d2dab58354270c60cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.208758 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b952c7cbc2447ab98c17bcb2f80c8ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.209057 22614 tablet_service.cc:1467] Processing CreateTablet for tablet 71d8cf6286aa4c3cb4f87d5974cf8548 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=3e579f694fc54bdba0aade1f5091d17b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:04.209291 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71d8cf6286aa4c3cb4f87d5974cf8548. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.211555 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 71d8cf6286aa4c3cb4f87d5974cf8548 (DEFAULT_TABLE table=rebalancer_extra_table_20 [id=3e579f694fc54bdba0aade1f5091d17b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:04.211830 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71d8cf6286aa4c3cb4f87d5974cf8548. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.211829 28162 consensus_queue.cc:1035] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:04.214056 28313 tablet_bootstrap.cc:492] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:04.215251 28313 tablet_bootstrap.cc:654] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:04.215782 28267 tablet_bootstrap.cc:492] T b7cdcee32c8f4cd3af30c402b8da2484 P 1aecef81733c47b3b468504934357231: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.216053 28309 tablet_bootstrap.cc:492] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:04.217005 28309 tablet_bootstrap.cc:654] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:04.217300 28267 tablet_bootstrap.cc:492] T b7cdcee32c8f4cd3af30c402b8da2484 P 1aecef81733c47b3b468504934357231: Bootstrap complete.
I20240502 07:59:04.217543 28267 ts_tablet_manager.cc:1367] T b7cdcee32c8f4cd3af30c402b8da2484 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.042s user 0.027s sys 0.000s
I20240502 07:59:04.217986 28267 raft_consensus.cc:348] T b7cdcee32c8f4cd3af30c402b8da2484 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:04.218111 28267 raft_consensus.cc:729] T b7cdcee32c8f4cd3af30c402b8da2484 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.218247 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:04.218801 28267 consensus_queue.cc:260] T b7cdcee32c8f4cd3af30c402b8da2484 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:04.219285 28267 ts_tablet_manager.cc:1398] T b7cdcee32c8f4cd3af30c402b8da2484 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:04.219792 22653 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session 1aecef81733c47b3b468504934357231-b7cdcee32c8f4cd3af30c402b8da2484 received from {username='slave'} at 127.15.129.134:44071
I20240502 07:59:04.219872 22653 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session 1aecef81733c47b3b468504934357231-b7cdcee32c8f4cd3af30c402b8da2484 on tablet b7cdcee32c8f4cd3af30c402b8da2484 with peer 1aecef81733c47b3b468504934357231
I20240502 07:59:04.220589 28303 tablet_copy_client.cc:523] T e3fd481f3c78497eb629b0180c07674f P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.222965 28303 tablet_bootstrap.cc:492] T e3fd481f3c78497eb629b0180c07674f P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:04.223866 28278 ts_tablet_manager.cc:902] T dbf60f171dfe4c88840587bf0df64cac P 43930dee0cd5498fa8e357a34dd71432: Initiating tablet copy from peer fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:04.225052 28278 tablet_copy_client.cc:308] T dbf60f171dfe4c88840587bf0df64cac P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.130:39625
I20240502 07:59:04.226081 28308 tablet_bootstrap.cc:492] T ecaf2063c4cb4d2dab58354270c60cb2 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:04.227527 28308 tablet_bootstrap.cc:654] T ecaf2063c4cb4d2dab58354270c60cb2 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:04.227977 22523 tablet_copy_service.cc:140] P fabddb16c85944648a710ec52fc7244c: Received BeginTabletCopySession request for tablet dbf60f171dfe4c88840587bf0df64cac from peer 43930dee0cd5498fa8e357a34dd71432 ({username='slave'} at 127.15.129.136:45143)
I20240502 07:59:04.228082 22523 tablet_copy_service.cc:161] P fabddb16c85944648a710ec52fc7244c: Beginning new tablet copy session on tablet dbf60f171dfe4c88840587bf0df64cac from peer 43930dee0cd5498fa8e357a34dd71432 at {username='slave'} at 127.15.129.136:45143: session id = 43930dee0cd5498fa8e357a34dd71432-dbf60f171dfe4c88840587bf0df64cac
I20240502 07:59:04.229192 22523 tablet_copy_source_session.cc:205] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:04.230343 28008 consensus_queue.cc:1035] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:04.235118 28303 tablet_bootstrap.cc:492] T e3fd481f3c78497eb629b0180c07674f P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.235462 28303 tablet_bootstrap.cc:492] T e3fd481f3c78497eb629b0180c07674f P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap complete.
I20240502 07:59:04.235663 28303 ts_tablet_manager.cc:1367] T e3fd481f3c78497eb629b0180c07674f P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20240502 07:59:04.236296 28303 raft_consensus.cc:348] T e3fd481f3c78497eb629b0180c07674f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.236516 28303 raft_consensus.cc:729] T e3fd481f3c78497eb629b0180c07674f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.236714 28303 consensus_queue.cc:260] T e3fd481f3c78497eb629b0180c07674f P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.237037 24932 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:04.238320 28278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf60f171dfe4c88840587bf0df64cac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.239933 28303 ts_tablet_manager.cc:1398] T e3fd481f3c78497eb629b0180c07674f P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:04.241523 28308 tablet_bootstrap.cc:492] T ecaf2063c4cb4d2dab58354270c60cb2 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:04.241576 25235 tablet_copy_service.cc:342] P 43930dee0cd5498fa8e357a34dd71432: Request end of tablet copy session e51b10af49484aaa8f0267aebc1bf27f-e3fd481f3c78497eb629b0180c07674f received from {username='slave'} at 127.15.129.131:60465
I20240502 07:59:04.241604 22483 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:04.241659 25235 tablet_copy_service.cc:433] P 43930dee0cd5498fa8e357a34dd71432: ending tablet copy session e51b10af49484aaa8f0267aebc1bf27f-e3fd481f3c78497eb629b0180c07674f on tablet e3fd481f3c78497eb629b0180c07674f with peer e51b10af49484aaa8f0267aebc1bf27f
I20240502 07:59:04.241674 28308 ts_tablet_manager.cc:1367] T ecaf2063c4cb4d2dab58354270c60cb2 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 07:59:04.242164 28308 raft_consensus.cc:348] T ecaf2063c4cb4d2dab58354270c60cb2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:04.242329 28308 raft_consensus.cc:374] T ecaf2063c4cb4d2dab58354270c60cb2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:04.242503 28308 raft_consensus.cc:729] T ecaf2063c4cb4d2dab58354270c60cb2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.242517 28278 tablet_copy_client.cc:783] T dbf60f171dfe4c88840587bf0df64cac P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:04.242668 28278 tablet_copy_client.cc:647] T dbf60f171dfe4c88840587bf0df64cac P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:04.242648 28308 consensus_queue.cc:260] T ecaf2063c4cb4d2dab58354270c60cb2 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:04.242990 28308 ts_tablet_manager.cc:1398] T ecaf2063c4cb4d2dab58354270c60cb2 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:04.258157 28325 ts_tablet_manager.cc:902] T af4d275b38ab49e7ac63e2fb9ffd8c07 P fabddb16c85944648a710ec52fc7244c: Initiating tablet copy from peer 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:04.258704 28309 tablet_bootstrap.cc:492] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:04.258929 28309 ts_tablet_manager.cc:1367] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.043s user 0.002s sys 0.000s
I20240502 07:59:04.259436 28309 raft_consensus.cc:348] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:04.259565 28309 raft_consensus.cc:374] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:04.259624 28309 raft_consensus.cc:729] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.259759 28309 consensus_queue.cc:260] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:04.261279 28309 ts_tablet_manager.cc:1398] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:04.261502 28309 tablet_bootstrap.cc:492] T 5b952c7cbc2447ab98c17bcb2f80c8ac P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:04.263974 28309 tablet_bootstrap.cc:654] T 5b952c7cbc2447ab98c17bcb2f80c8ac P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:04.265614 28325 tablet_copy_client.cc:308] T af4d275b38ab49e7ac63e2fb9ffd8c07 P fabddb16c85944648a710ec52fc7244c: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.136:36591
I20240502 07:59:04.266206 25235 tablet_copy_service.cc:140] P 43930dee0cd5498fa8e357a34dd71432: Received BeginTabletCopySession request for tablet af4d275b38ab49e7ac63e2fb9ffd8c07 from peer fabddb16c85944648a710ec52fc7244c ({username='slave'} at 127.15.129.130:50787)
I20240502 07:59:04.266304 25235 tablet_copy_service.cc:161] P 43930dee0cd5498fa8e357a34dd71432: Beginning new tablet copy session on tablet af4d275b38ab49e7ac63e2fb9ffd8c07 from peer fabddb16c85944648a710ec52fc7244c at {username='slave'} at 127.15.129.130:50787: session id = fabddb16c85944648a710ec52fc7244c-af4d275b38ab49e7ac63e2fb9ffd8c07
I20240502 07:59:04.266716 28273 tablet_bootstrap.cc:492] T b270d89546214f4d9d15193f4f8e04d5 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.267014 28273 tablet_bootstrap.cc:492] T b270d89546214f4d9d15193f4f8e04d5 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap complete.
I20240502 07:59:04.267244 25235 tablet_copy_source_session.cc:205] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:04.267356 28273 ts_tablet_manager.cc:1367] T b270d89546214f4d9d15193f4f8e04d5 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.122s user 0.028s sys 0.000s
I20240502 07:59:04.267822 28273 raft_consensus.cc:348] T b270d89546214f4d9d15193f4f8e04d5 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:04.267953 28273 raft_consensus.cc:729] T b270d89546214f4d9d15193f4f8e04d5 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.268118 28273 consensus_queue.cc:260] T b270d89546214f4d9d15193f4f8e04d5 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:04.268517 28273 ts_tablet_manager.cc:1398] T b270d89546214f4d9d15193f4f8e04d5 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:04.268688 28325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af4d275b38ab49e7ac63e2fb9ffd8c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.271564 22653 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-b270d89546214f4d9d15193f4f8e04d5 received from {username='slave'} at 127.15.129.133:55755
I20240502 07:59:04.271656 22653 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-b270d89546214f4d9d15193f4f8e04d5 on tablet b270d89546214f4d9d15193f4f8e04d5 with peer 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:04.272089 28325 tablet_copy_client.cc:783] T af4d275b38ab49e7ac63e2fb9ffd8c07 P fabddb16c85944648a710ec52fc7244c: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:04.272269 28325 tablet_copy_client.cc:647] T af4d275b38ab49e7ac63e2fb9ffd8c07 P fabddb16c85944648a710ec52fc7244c: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:04.274714 28311 tablet_bootstrap.cc:492] T 5b952c7cbc2447ab98c17bcb2f80c8ac P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:04.274762 28309 tablet_bootstrap.cc:492] T 5b952c7cbc2447ab98c17bcb2f80c8ac P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:04.274778 28325 tablet_copy_client.cc:523] T af4d275b38ab49e7ac63e2fb9ffd8c07 P fabddb16c85944648a710ec52fc7244c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.274856 28309 ts_tablet_manager.cc:1367] T 5b952c7cbc2447ab98c17bcb2f80c8ac P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.002s
I20240502 07:59:04.275595 28309 raft_consensus.cc:348] T 5b952c7cbc2447ab98c17bcb2f80c8ac P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.275792 28309 raft_consensus.cc:374] T 5b952c7cbc2447ab98c17bcb2f80c8ac P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:04.275875 28309 raft_consensus.cc:729] T 5b952c7cbc2447ab98c17bcb2f80c8ac P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.276065 28309 consensus_queue.cc:260] T 5b952c7cbc2447ab98c17bcb2f80c8ac P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.276517 28325 tablet_bootstrap.cc:492] T af4d275b38ab49e7ac63e2fb9ffd8c07 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:04.277024 28311 tablet_bootstrap.cc:654] T 5b952c7cbc2447ab98c17bcb2f80c8ac P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:04.278368 28310 tablet_bootstrap.cc:492] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:04.278846 22352 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:04.279088 28309 ts_tablet_manager.cc:1398] T 5b952c7cbc2447ab98c17bcb2f80c8ac P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240502 07:59:04.279098 25063 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:04.280200 22225 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:04.281450 28310 tablet_bootstrap.cc:654] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:04.282817 28314 tablet_bootstrap.cc:492] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:04.283844 28314 tablet_bootstrap.cc:654] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:04.288450 28314 tablet_bootstrap.cc:492] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:04.288579 28314 ts_tablet_manager.cc:1367] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:04.289013 28314 raft_consensus.cc:348] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.289137 28314 raft_consensus.cc:374] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:04.290740 28311 tablet_bootstrap.cc:492] T 5b952c7cbc2447ab98c17bcb2f80c8ac P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:04.291590 28310 tablet_bootstrap.cc:492] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:04.291682 28278 tablet_copy_client.cc:523] T dbf60f171dfe4c88840587bf0df64cac P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.291739 28310 ts_tablet_manager.cc:1367] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.001s
I20240502 07:59:04.292562 28296 tablet.cc:1722] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:04.292847 28296 tablet.cc:1620] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:04.292969 28311 ts_tablet_manager.cc:1367] T 5b952c7cbc2447ab98c17bcb2f80c8ac P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 07:59:04.293221 28278 tablet_bootstrap.cc:492] T dbf60f171dfe4c88840587bf0df64cac P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:04.293622 28311 raft_consensus.cc:348] T 5b952c7cbc2447ab98c17bcb2f80c8ac P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.293738 28311 raft_consensus.cc:374] T 5b952c7cbc2447ab98c17bcb2f80c8ac P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:04.293772 28311 raft_consensus.cc:729] T 5b952c7cbc2447ab98c17bcb2f80c8ac P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.293900 28311 consensus_queue.cc:260] T 5b952c7cbc2447ab98c17bcb2f80c8ac P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.294028 28314 raft_consensus.cc:729] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.295209 28314 consensus_queue.cc:260] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.295491 28295 tablet.cc:1722] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:04.295768 28295 tablet.cc:1620] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:04.296257 28314 ts_tablet_manager.cc:1398] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20240502 07:59:04.296633 28311 ts_tablet_manager.cc:1398] T 5b952c7cbc2447ab98c17bcb2f80c8ac P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:04.292788 28310 raft_consensus.cc:348] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.296718 28310 raft_consensus.cc:374] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:04.296761 28310 raft_consensus.cc:729] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.296911 28310 consensus_queue.cc:260] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.300583 28303 ts_tablet_manager.cc:902] T f7828cb273c64934853b2cd38368402f P e51b10af49484aaa8f0267aebc1bf27f: Initiating tablet copy from peer 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:04.301131 28303 tablet_copy_client.cc:308] T f7828cb273c64934853b2cd38368402f P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.135:42733
I20240502 07:59:04.302716 28311 tablet_bootstrap.cc:492] T ecaf2063c4cb4d2dab58354270c60cb2 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:04.303241 28325 tablet_bootstrap.cc:492] T af4d275b38ab49e7ac63e2fb9ffd8c07 P fabddb16c85944648a710ec52fc7244c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.303495 28310 ts_tablet_manager.cc:1398] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.012s user 0.000s sys 0.002s
I20240502 07:59:04.303619 28325 tablet_bootstrap.cc:492] T af4d275b38ab49e7ac63e2fb9ffd8c07 P fabddb16c85944648a710ec52fc7244c: Bootstrap complete.
I20240502 07:59:04.303818 28325 ts_tablet_manager.cc:1367] T af4d275b38ab49e7ac63e2fb9ffd8c07 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.027s user 0.004s sys 0.001s
I20240502 07:59:04.304450 28311 tablet_bootstrap.cc:654] T ecaf2063c4cb4d2dab58354270c60cb2 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:04.304906 25104 tablet_copy_service.cc:140] P 227eec4b5f014d349661f6e0b39b9055: Received BeginTabletCopySession request for tablet f7828cb273c64934853b2cd38368402f from peer e51b10af49484aaa8f0267aebc1bf27f ({username='slave'} at 127.15.129.131:57623)
I20240502 07:59:04.305065 25104 tablet_copy_service.cc:161] P 227eec4b5f014d349661f6e0b39b9055: Beginning new tablet copy session on tablet f7828cb273c64934853b2cd38368402f from peer e51b10af49484aaa8f0267aebc1bf27f at {username='slave'} at 127.15.129.131:57623: session id = e51b10af49484aaa8f0267aebc1bf27f-f7828cb273c64934853b2cd38368402f
I20240502 07:59:04.306473 28301 tablet.cc:1722] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:04.306556 25104 tablet_copy_source_session.cc:205] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:04.306775 28301 tablet.cc:1620] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:04.304356 28325 raft_consensus.cc:348] T af4d275b38ab49e7ac63e2fb9ffd8c07 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:04.307754 28303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7828cb273c64934853b2cd38368402f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.309162 28325 raft_consensus.cc:729] T af4d275b38ab49e7ac63e2fb9ffd8c07 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.309350 28325 consensus_queue.cc:260] T af4d275b38ab49e7ac63e2fb9ffd8c07 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:04.309762 28325 ts_tablet_manager.cc:1398] T af4d275b38ab49e7ac63e2fb9ffd8c07 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20240502 07:59:04.309878 28313 tablet_bootstrap.cc:492] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:04.309973 28313 ts_tablet_manager.cc:1367] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.096s user 0.002s sys 0.000s
I20240502 07:59:04.310432 28313 raft_consensus.cc:348] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.310578 28313 raft_consensus.cc:374] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:04.310611 28313 raft_consensus.cc:729] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.310751 28313 consensus_queue.cc:260] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.310915 28303 tablet_copy_client.cc:783] T f7828cb273c64934853b2cd38368402f P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:04.311031 28313 ts_tablet_manager.cc:1398] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:04.311050 28303 tablet_copy_client.cc:647] T f7828cb273c64934853b2cd38368402f P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:04.311771 22223 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=fb52db55839b4ca6b11a412261f5d725] alter complete (version 2)
I20240502 07:59:04.311911 28340 ts_tablet_manager.cc:902] T 288b33b7a5f84af59bde5fae34cbe924 P fabddb16c85944648a710ec52fc7244c: Initiating tablet copy from peer 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:04.312585 25235 tablet_copy_service.cc:342] P 43930dee0cd5498fa8e357a34dd71432: Request end of tablet copy session fabddb16c85944648a710ec52fc7244c-af4d275b38ab49e7ac63e2fb9ffd8c07 received from {username='slave'} at 127.15.129.130:50787
I20240502 07:59:04.312664 25235 tablet_copy_service.cc:433] P 43930dee0cd5498fa8e357a34dd71432: ending tablet copy session fabddb16c85944648a710ec52fc7244c-af4d275b38ab49e7ac63e2fb9ffd8c07 on tablet af4d275b38ab49e7ac63e2fb9ffd8c07 with peer fabddb16c85944648a710ec52fc7244c
I20240502 07:59:04.313886 28340 tablet_copy_client.cc:308] T 288b33b7a5f84af59bde5fae34cbe924 P fabddb16c85944648a710ec52fc7244c: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.134:35995
I20240502 07:59:04.314425 24972 tablet_copy_service.cc:140] P 1aecef81733c47b3b468504934357231: Received BeginTabletCopySession request for tablet 288b33b7a5f84af59bde5fae34cbe924 from peer fabddb16c85944648a710ec52fc7244c ({username='slave'} at 127.15.129.130:52777)
I20240502 07:59:04.314545 24972 tablet_copy_service.cc:161] P 1aecef81733c47b3b468504934357231: Beginning new tablet copy session on tablet 288b33b7a5f84af59bde5fae34cbe924 from peer fabddb16c85944648a710ec52fc7244c at {username='slave'} at 127.15.129.130:52777: session id = fabddb16c85944648a710ec52fc7244c-288b33b7a5f84af59bde5fae34cbe924
I20240502 07:59:04.315506 24972 tablet_copy_source_session.cc:205] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:04.317263 28340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 288b33b7a5f84af59bde5fae34cbe924. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.318509 28303 tablet_copy_client.cc:523] T f7828cb273c64934853b2cd38368402f P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.318565 28273 ts_tablet_manager.cc:902] T 9f44c905bfaa4db093a1b39b6c21e03c P 4b4e7eeb7f3f42119c0363a79febc54b: Initiating tablet copy from peer 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:04.318989 28273 tablet_copy_client.cc:308] T 9f44c905bfaa4db093a1b39b6c21e03c P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.135:42733
I20240502 07:59:04.319465 25104 tablet_copy_service.cc:140] P 227eec4b5f014d349661f6e0b39b9055: Received BeginTabletCopySession request for tablet 9f44c905bfaa4db093a1b39b6c21e03c from peer 4b4e7eeb7f3f42119c0363a79febc54b ({username='slave'} at 127.15.129.133:40231)
I20240502 07:59:04.319561 25104 tablet_copy_service.cc:161] P 227eec4b5f014d349661f6e0b39b9055: Beginning new tablet copy session on tablet 9f44c905bfaa4db093a1b39b6c21e03c from peer 4b4e7eeb7f3f42119c0363a79febc54b at {username='slave'} at 127.15.129.133:40231: session id = 4b4e7eeb7f3f42119c0363a79febc54b-9f44c905bfaa4db093a1b39b6c21e03c
I20240502 07:59:04.319900 28303 tablet_bootstrap.cc:492] T f7828cb273c64934853b2cd38368402f P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:04.320215 28340 tablet_copy_client.cc:783] T 288b33b7a5f84af59bde5fae34cbe924 P fabddb16c85944648a710ec52fc7244c: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:04.320458 28340 tablet_copy_client.cc:647] T 288b33b7a5f84af59bde5fae34cbe924 P fabddb16c85944648a710ec52fc7244c: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:04.320520 25104 tablet_copy_source_session.cc:205] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:04.321393 28273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f44c905bfaa4db093a1b39b6c21e03c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.324239 28273 tablet_copy_client.cc:783] T 9f44c905bfaa4db093a1b39b6c21e03c P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:04.324438 28273 tablet_copy_client.cc:647] T 9f44c905bfaa4db093a1b39b6c21e03c P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:04.327136 28273 tablet_copy_client.cc:523] T 9f44c905bfaa4db093a1b39b6c21e03c P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.327890 28311 tablet_bootstrap.cc:492] T ecaf2063c4cb4d2dab58354270c60cb2 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:04.327983 28311 ts_tablet_manager.cc:1367] T ecaf2063c4cb4d2dab58354270c60cb2 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20240502 07:59:04.328653 28311 raft_consensus.cc:348] T ecaf2063c4cb4d2dab58354270c60cb2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:04.328769 28311 raft_consensus.cc:374] T ecaf2063c4cb4d2dab58354270c60cb2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:04.328804 28311 raft_consensus.cc:729] T ecaf2063c4cb4d2dab58354270c60cb2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.328924 28311 consensus_queue.cc:260] T ecaf2063c4cb4d2dab58354270c60cb2 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:04.328990 28273 tablet_bootstrap.cc:492] T 9f44c905bfaa4db093a1b39b6c21e03c P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:04.329222 28311 ts_tablet_manager.cc:1398] T ecaf2063c4cb4d2dab58354270c60cb2 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:04.330271 28311 tablet_bootstrap.cc:492] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:04.332422 28311 tablet_bootstrap.cc:654] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:04.333456 28311 tablet_bootstrap.cc:492] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:04.333603 28311 ts_tablet_manager.cc:1367] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:04.334053 28311 raft_consensus.cc:348] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.334239 28311 raft_consensus.cc:374] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:04.334318 28311 raft_consensus.cc:729] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.334537 28311 consensus_queue.cc:260] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.334911 28311 ts_tablet_manager.cc:1398] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:04.335083 28273 tablet_bootstrap.cc:492] T 9f44c905bfaa4db093a1b39b6c21e03c P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.336774 28273 tablet_bootstrap.cc:492] T 9f44c905bfaa4db093a1b39b6c21e03c P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap complete.
I20240502 07:59:04.336949 28273 ts_tablet_manager.cc:1367] T 9f44c905bfaa4db093a1b39b6c21e03c P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20240502 07:59:04.337062 28340 tablet_copy_client.cc:523] T 288b33b7a5f84af59bde5fae34cbe924 P fabddb16c85944648a710ec52fc7244c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.338598 28340 tablet_bootstrap.cc:492] T 288b33b7a5f84af59bde5fae34cbe924 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:04.338547 28273 raft_consensus.cc:348] T 9f44c905bfaa4db093a1b39b6c21e03c P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:04.338662 28273 raft_consensus.cc:729] T 9f44c905bfaa4db093a1b39b6c21e03c P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.338821 28273 consensus_queue.cc:260] T 9f44c905bfaa4db093a1b39b6c21e03c P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:04.339615 28273 ts_tablet_manager.cc:1398] T 9f44c905bfaa4db093a1b39b6c21e03c P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:04.340273 25104 tablet_copy_service.cc:342] P 227eec4b5f014d349661f6e0b39b9055: Request end of tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-9f44c905bfaa4db093a1b39b6c21e03c received from {username='slave'} at 127.15.129.133:40231
I20240502 07:59:04.340382 25104 tablet_copy_service.cc:433] P 227eec4b5f014d349661f6e0b39b9055: ending tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-9f44c905bfaa4db093a1b39b6c21e03c on tablet 9f44c905bfaa4db093a1b39b6c21e03c with peer 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:04.344509 28340 tablet_bootstrap.cc:492] T 288b33b7a5f84af59bde5fae34cbe924 P fabddb16c85944648a710ec52fc7244c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.344770 28340 tablet_bootstrap.cc:492] T 288b33b7a5f84af59bde5fae34cbe924 P fabddb16c85944648a710ec52fc7244c: Bootstrap complete.
I20240502 07:59:04.344939 28340 ts_tablet_manager.cc:1367] T 288b33b7a5f84af59bde5fae34cbe924 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20240502 07:59:04.345840 28354 ts_tablet_manager.cc:902] T cc97164c8cfe4f91afabcc606d9821ef P e51b10af49484aaa8f0267aebc1bf27f: Initiating tablet copy from peer fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:04.345973 28340 raft_consensus.cc:348] T 288b33b7a5f84af59bde5fae34cbe924 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:04.346096 28340 raft_consensus.cc:729] T 288b33b7a5f84af59bde5fae34cbe924 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.346235 28340 consensus_queue.cc:260] T 288b33b7a5f84af59bde5fae34cbe924 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:04.346778 28354 tablet_copy_client.cc:308] T cc97164c8cfe4f91afabcc606d9821ef P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.130:39625
I20240502 07:59:04.347290 28340 ts_tablet_manager.cc:1398] T 288b33b7a5f84af59bde5fae34cbe924 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20240502 07:59:04.347375 22523 tablet_copy_service.cc:140] P fabddb16c85944648a710ec52fc7244c: Received BeginTabletCopySession request for tablet cc97164c8cfe4f91afabcc606d9821ef from peer e51b10af49484aaa8f0267aebc1bf27f ({username='slave'} at 127.15.129.131:46727)
I20240502 07:59:04.347443 22523 tablet_copy_service.cc:161] P fabddb16c85944648a710ec52fc7244c: Beginning new tablet copy session on tablet cc97164c8cfe4f91afabcc606d9821ef from peer e51b10af49484aaa8f0267aebc1bf27f at {username='slave'} at 127.15.129.131:46727: session id = e51b10af49484aaa8f0267aebc1bf27f-cc97164c8cfe4f91afabcc606d9821ef
I20240502 07:59:04.348320 28303 tablet_bootstrap.cc:492] T f7828cb273c64934853b2cd38368402f P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.348420 22523 tablet_copy_source_session.cc:205] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:04.348641 28303 tablet_bootstrap.cc:492] T f7828cb273c64934853b2cd38368402f P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap complete.
I20240502 07:59:04.348843 28303 ts_tablet_manager.cc:1367] T f7828cb273c64934853b2cd38368402f P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.029s user 0.005s sys 0.000s
I20240502 07:59:04.349067 24972 tablet_copy_service.cc:342] P 1aecef81733c47b3b468504934357231: Request end of tablet copy session fabddb16c85944648a710ec52fc7244c-288b33b7a5f84af59bde5fae34cbe924 received from {username='slave'} at 127.15.129.130:52777
I20240502 07:59:04.349148 24972 tablet_copy_service.cc:433] P 1aecef81733c47b3b468504934357231: ending tablet copy session fabddb16c85944648a710ec52fc7244c-288b33b7a5f84af59bde5fae34cbe924 on tablet 288b33b7a5f84af59bde5fae34cbe924 with peer fabddb16c85944648a710ec52fc7244c
I20240502 07:59:04.349367 28303 raft_consensus.cc:348] T f7828cb273c64934853b2cd38368402f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.349493 28303 raft_consensus.cc:729] T f7828cb273c64934853b2cd38368402f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.349684 28303 consensus_queue.cc:260] T f7828cb273c64934853b2cd38368402f P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.349764 28354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc97164c8cfe4f91afabcc606d9821ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.350015 28303 ts_tablet_manager.cc:1398] T f7828cb273c64934853b2cd38368402f P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:04.350435 25104 tablet_copy_service.cc:342] P 227eec4b5f014d349661f6e0b39b9055: Request end of tablet copy session e51b10af49484aaa8f0267aebc1bf27f-f7828cb273c64934853b2cd38368402f received from {username='slave'} at 127.15.129.131:57623
I20240502 07:59:04.350540 25104 tablet_copy_service.cc:433] P 227eec4b5f014d349661f6e0b39b9055: ending tablet copy session e51b10af49484aaa8f0267aebc1bf27f-f7828cb273c64934853b2cd38368402f on tablet f7828cb273c64934853b2cd38368402f with peer e51b10af49484aaa8f0267aebc1bf27f
I20240502 07:59:04.354118 28354 tablet_copy_client.cc:783] T cc97164c8cfe4f91afabcc606d9821ef P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:04.354341 28354 tablet_copy_client.cc:647] T cc97164c8cfe4f91afabcc606d9821ef P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:04.357528 28354 tablet_copy_client.cc:523] T cc97164c8cfe4f91afabcc606d9821ef P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:04.358877 28354 tablet_bootstrap.cc:492] T cc97164c8cfe4f91afabcc606d9821ef P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:04.361980 28354 tablet_bootstrap.cc:492] T cc97164c8cfe4f91afabcc606d9821ef P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.362201 28354 tablet_bootstrap.cc:492] T cc97164c8cfe4f91afabcc606d9821ef P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap complete.
I20240502 07:59:04.362344 28354 ts_tablet_manager.cc:1367] T cc97164c8cfe4f91afabcc606d9821ef P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.002s
I20240502 07:59:04.362702 28354 raft_consensus.cc:348] T cc97164c8cfe4f91afabcc606d9821ef P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.362803 28354 raft_consensus.cc:729] T cc97164c8cfe4f91afabcc606d9821ef P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.362949 28354 consensus_queue.cc:260] T cc97164c8cfe4f91afabcc606d9821ef P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.363235 28354 ts_tablet_manager.cc:1398] T cc97164c8cfe4f91afabcc606d9821ef P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:04.363828 22523 tablet_copy_service.cc:342] P fabddb16c85944648a710ec52fc7244c: Request end of tablet copy session e51b10af49484aaa8f0267aebc1bf27f-cc97164c8cfe4f91afabcc606d9821ef received from {username='slave'} at 127.15.129.131:46727
I20240502 07:59:04.363912 22523 tablet_copy_service.cc:433] P fabddb16c85944648a710ec52fc7244c: ending tablet copy session e51b10af49484aaa8f0267aebc1bf27f-cc97164c8cfe4f91afabcc606d9821ef on tablet cc97164c8cfe4f91afabcc606d9821ef with peer e51b10af49484aaa8f0267aebc1bf27f
I20240502 07:59:04.363905 28278 tablet_bootstrap.cc:492] T dbf60f171dfe4c88840587bf0df64cac P 43930dee0cd5498fa8e357a34dd71432: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:04.364171 28278 tablet_bootstrap.cc:492] T dbf60f171dfe4c88840587bf0df64cac P 43930dee0cd5498fa8e357a34dd71432: Bootstrap complete.
I20240502 07:59:04.364336 28278 ts_tablet_manager.cc:1367] T dbf60f171dfe4c88840587bf0df64cac P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.071s user 0.024s sys 0.000s
I20240502 07:59:04.364740 28278 raft_consensus.cc:348] T dbf60f171dfe4c88840587bf0df64cac P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:04.364852 28278 raft_consensus.cc:729] T dbf60f171dfe4c88840587bf0df64cac P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.365002 28278 consensus_queue.cc:260] T dbf60f171dfe4c88840587bf0df64cac P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:04.365943 28278 ts_tablet_manager.cc:1398] T dbf60f171dfe4c88840587bf0df64cac P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:04.366392 22523 tablet_copy_service.cc:342] P fabddb16c85944648a710ec52fc7244c: Request end of tablet copy session 43930dee0cd5498fa8e357a34dd71432-dbf60f171dfe4c88840587bf0df64cac received from {username='slave'} at 127.15.129.136:45143
I20240502 07:59:04.366472 22523 tablet_copy_service.cc:433] P fabddb16c85944648a710ec52fc7244c: ending tablet copy session 43930dee0cd5498fa8e357a34dd71432-dbf60f171dfe4c88840587bf0df64cac on tablet dbf60f171dfe4c88840587bf0df64cac with peer 43930dee0cd5498fa8e357a34dd71432
I20240502 07:59:04.397768 27752 raft_consensus.cc:482] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:04.397893 27752 raft_consensus.cc:504] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:04.398247 27752 leader_election.cc:288] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:04.398639 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecaf2063c4cb4d2dab58354270c60cb2" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:04.398670 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecaf2063c4cb4d2dab58354270c60cb2" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:04.398785 22634 raft_consensus.cc:2454] T ecaf2063c4cb4d2dab58354270c60cb2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:04.398808 24952 raft_consensus.cc:2454] T ecaf2063c4cb4d2dab58354270c60cb2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:04.399502 25151 leader_election.cc:302] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:04.399681 27752 raft_consensus.cc:2781] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:04.399739 27752 raft_consensus.cc:482] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:04.399780 27752 raft_consensus.cc:3037] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:04.400593 27752 raft_consensus.cc:504] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:04.400890 27752 leader_election.cc:288] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:04.401187 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecaf2063c4cb4d2dab58354270c60cb2" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:04.401312 22634 raft_consensus.cc:3037] T ecaf2063c4cb4d2dab58354270c60cb2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:04.401474 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecaf2063c4cb4d2dab58354270c60cb2" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:04.401599 24952 raft_consensus.cc:3037] T ecaf2063c4cb4d2dab58354270c60cb2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:04.402280 22634 raft_consensus.cc:2454] T ecaf2063c4cb4d2dab58354270c60cb2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:04.402438 24952 raft_consensus.cc:2454] T ecaf2063c4cb4d2dab58354270c60cb2 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:04.402570 25151 leader_election.cc:302] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:04.402933 27752 raft_consensus.cc:2781] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:04.403026 27752 raft_consensus.cc:686] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:04.403183 27752 consensus_queue.cc:237] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:04.405124 22226 catalog_manager.cc:5484] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:04.418711 22228 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:04.425076 28295 tablet.cc:1722] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:04.426968 28301 tablet.cc:1722] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:04.433586 28296 tablet.cc:1722] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:04.434474 22227 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=fb52db55839b4ca6b11a412261f5d725] alter complete (version 3)
I20240502 07:59:04.444240 28015 raft_consensus.cc:482] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:04.444379 28015 raft_consensus.cc:504] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.444903 28015 leader_election.cc:288] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:04.445884 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b952c7cbc2447ab98c17bcb2f80c8ac" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:04.446017 22634 raft_consensus.cc:2454] T 5b952c7cbc2447ab98c17bcb2f80c8ac P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:04.446310 22308 leader_election.cc:302] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:04.446496 28015 raft_consensus.cc:2781] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:04.446563 28015 raft_consensus.cc:482] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:04.446606 28015 raft_consensus.cc:3037] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:04.447868 28015 raft_consensus.cc:504] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.448242 28015 leader_election.cc:288] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:04.448781 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b952c7cbc2447ab98c17bcb2f80c8ac" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:04.448922 22634 raft_consensus.cc:3037] T 5b952c7cbc2447ab98c17bcb2f80c8ac P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:04.450052 22634 raft_consensus.cc:2454] T 5b952c7cbc2447ab98c17bcb2f80c8ac P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:04.451828 22308 leader_election.cc:302] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:04.452569 28015 raft_consensus.cc:2781] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:04.452739 28015 raft_consensus.cc:686] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:59:04.452698 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b952c7cbc2447ab98c17bcb2f80c8ac" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:04.452836 25214 raft_consensus.cc:3037] T 5b952c7cbc2447ab98c17bcb2f80c8ac P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:04.452889 28015 consensus_queue.cc:237] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.453859 25214 raft_consensus.cc:2454] T 5b952c7cbc2447ab98c17bcb2f80c8ac P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:04.455400 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b952c7cbc2447ab98c17bcb2f80c8ac" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:04.455547 25215 raft_consensus.cc:2362] T 5b952c7cbc2447ab98c17bcb2f80c8ac P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1. Re-sending same reply.
I20240502 07:59:04.458344 22228 catalog_manager.cc:5484] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:04.512598 27288 raft_consensus.cc:482] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:04.512727 27288 raft_consensus.cc:504] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.513177 27288 leader_election.cc:288] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:04.513559 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71d8cf6286aa4c3cb4f87d5974cf8548" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:04.513566 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71d8cf6286aa4c3cb4f87d5974cf8548" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:04.513703 24820 raft_consensus.cc:2454] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:04.513715 25084 raft_consensus.cc:2454] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:04.513999 22569 leader_election.cc:302] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:04.514206 27288 raft_consensus.cc:2781] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:04.514262 27288 raft_consensus.cc:482] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:04.514308 27288 raft_consensus.cc:3037] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:04.515612 27288 raft_consensus.cc:504] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.516136 27288 leader_election.cc:288] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:04.516407 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71d8cf6286aa4c3cb4f87d5974cf8548" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:04.516444 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71d8cf6286aa4c3cb4f87d5974cf8548" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:04.516527 24820 raft_consensus.cc:3037] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:04.516570 25084 raft_consensus.cc:3037] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:04.517381 25084 raft_consensus.cc:2454] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:04.517465 24820 raft_consensus.cc:2454] T 71d8cf6286aa4c3cb4f87d5974cf8548 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:04.517752 22567 leader_election.cc:302] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:04.517915 27965 raft_consensus.cc:2781] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:04.517994 27965 raft_consensus.cc:686] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:59:04.518131 27965 consensus_queue.cc:237] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.519878 22227 catalog_manager.cc:5484] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:04.538378 24820 raft_consensus.cc:1203] T b270d89546214f4d9d15193f4f8e04d5 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:04.538499 22227 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:04.538631 22227 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:04.544692 22227 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 7eaa8fb0cbc2467a9f512e6892bdb777
I20240502 07:59:04.545621 25195 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eaa8fb0cbc2467a9f512e6892bdb777 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:04 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:04.545755 24932 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eaa8fb0cbc2467a9f512e6892bdb777 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:04 UTC) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:04.545797 22352 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eaa8fb0cbc2467a9f512e6892bdb777 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:04 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:04.546939 28417 tablet_replica.cc:330] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:04.546989 24799 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:04.547523 28417 raft_consensus.cc:2229] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:04.547669 28417 raft_consensus.cc:2258] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:04.547951 28418 tablet_replica.cc:330] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:04.548101 28418 raft_consensus.cc:2229] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:04.548730 28417 ts_tablet_manager.cc:1875] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:04.549417 28418 raft_consensus.cc:2258] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:04.549958 28418 ts_tablet_manager.cc:1875] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:04.552920 28418 ts_tablet_manager.cc:1888] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:04.553170 28418 log.cc:1196] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/7eaa8fb0cbc2467a9f512e6892bdb777
I20240502 07:59:04.553493 28418 ts_tablet_manager.cc:1909] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 1aecef81733c47b3b468504934357231: Deleting consensus metadata
I20240502 07:59:04.554764 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet 7eaa8fb0cbc2467a9f512e6892bdb777 (table rebalancer_dynamic_table_new_name [id=fb52db55839b4ca6b11a412261f5d725]) successfully deleted
I20240502 07:59:04.555578 22352 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:04.555872 28417 ts_tablet_manager.cc:1888] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:04.555946 28417 log.cc:1196] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/7eaa8fb0cbc2467a9f512e6892bdb777
I20240502 07:59:04.556233 28417 ts_tablet_manager.cc:1909] T 7eaa8fb0cbc2467a9f512e6892bdb777 P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:59:04.557524 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 7eaa8fb0cbc2467a9f512e6892bdb777 (table rebalancer_dynamic_table_new_name [id=fb52db55839b4ca6b11a412261f5d725]) successfully deleted
I20240502 07:59:04.558696 25063 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:04.559535 24932 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:04.561103 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:04.566715 28419 tablet_replica.cc:330] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:04.566929 28419 raft_consensus.cc:2229] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:04.567113 28419 raft_consensus.cc:2258] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:04.567821 28419 ts_tablet_manager.cc:1875] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:04.572050 22483 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:04.572080 28419 ts_tablet_manager.cc:1888] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:04.575181 28419 log.cc:1196] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/7eaa8fb0cbc2467a9f512e6892bdb777
I20240502 07:59:04.575605 28419 ts_tablet_manager.cc:1909] T 7eaa8fb0cbc2467a9f512e6892bdb777 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:59:04.580606 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 7eaa8fb0cbc2467a9f512e6892bdb777 (table rebalancer_dynamic_table_new_name [id=fb52db55839b4ca6b11a412261f5d725]) successfully deleted
I20240502 07:59:04.582162 25195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:04.582679 22614 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:04.614187 22634 raft_consensus.cc:1203] T e3fd481f3c78497eb629b0180c07674f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:04.647099 22227 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:04.652319 25195 tablet_service.cc:1467] Processing CreateTablet for tablet 4420bb923c86440b8684cc6d23d98cf6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=ef0cc0fa74c142299578e1778a636bca]), partition=
I20240502 07:59:04.652343 24799 tablet_service.cc:1467] Processing CreateTablet for tablet 4420bb923c86440b8684cc6d23d98cf6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=ef0cc0fa74c142299578e1778a636bca]), partition=
I20240502 07:59:04.652626 24799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4420bb923c86440b8684cc6d23d98cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.652622 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4420bb923c86440b8684cc6d23d98cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.653771 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 4420bb923c86440b8684cc6d23d98cf6 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=ef0cc0fa74c142299578e1778a636bca]), partition=
I20240502 07:59:04.654006 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4420bb923c86440b8684cc6d23d98cf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:04.654784 28313 tablet_bootstrap.cc:492] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:04.655755 28313 tablet_bootstrap.cc:654] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:04.656114 28314 tablet_bootstrap.cc:492] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:04.656492 28309 tablet_bootstrap.cc:492] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:04.657105 28314 tablet_bootstrap.cc:654] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:04.657685 28309 tablet_bootstrap.cc:654] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:04.658475 28314 tablet_bootstrap.cc:492] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:04.658566 28314 ts_tablet_manager.cc:1367] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 07:59:04.659016 28314 raft_consensus.cc:348] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:04.659148 28314 raft_consensus.cc:374] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:04.659180 28314 raft_consensus.cc:729] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.659343 28313 tablet_bootstrap.cc:492] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:04.659309 28314 consensus_queue.cc:260] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:04.659427 28313 ts_tablet_manager.cc:1367] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:04.659641 28314 ts_tablet_manager.cc:1398] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:04.659889 28313 raft_consensus.cc:348] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:04.659993 28313 raft_consensus.cc:374] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:04.660024 28313 raft_consensus.cc:729] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.660133 28313 consensus_queue.cc:260] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:04.660398 28313 ts_tablet_manager.cc:1398] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:04.660574 24952 raft_consensus.cc:1203] T b7cdcee32c8f4cd3af30c402b8da2484 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:04.662678 25215 raft_consensus.cc:1203] T dbf60f171dfe4c88840587bf0df64cac P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:04.665652 28309 tablet_bootstrap.cc:492] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:04.665751 28309 ts_tablet_manager.cc:1367] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20240502 07:59:04.666234 28309 raft_consensus.cc:348] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:04.666347 28309 raft_consensus.cc:374] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:04.666380 28309 raft_consensus.cc:729] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:04.666500 28309 consensus_queue.cc:260] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:04.666769 28309 ts_tablet_manager.cc:1398] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:04.684871 24952 raft_consensus.cc:1203] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:04.713840 27323 raft_consensus.cc:482] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:04.713958 27323 raft_consensus.cc:504] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:04.714371 27323 leader_election.cc:288] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:04.714880 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4420bb923c86440b8684cc6d23d98cf6" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:04.715029 24820 raft_consensus.cc:2454] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:04.715416 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4420bb923c86440b8684cc6d23d98cf6" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:04.715508 25019 leader_election.cc:302] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:04.715559 25215 raft_consensus.cc:2454] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:04.715677 27323 raft_consensus.cc:2781] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:04.715735 27323 raft_consensus.cc:482] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:04.715773 27323 raft_consensus.cc:3037] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:04.716928 27323 raft_consensus.cc:504] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:04.717295 27323 leader_election.cc:288] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:04.717865 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4420bb923c86440b8684cc6d23d98cf6" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:04.717994 25215 raft_consensus.cc:3037] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:04.718211 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4420bb923c86440b8684cc6d23d98cf6" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:04.718325 24820 raft_consensus.cc:3037] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:04.719046 25215 raft_consensus.cc:2454] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:04.719246 24820 raft_consensus.cc:2454] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:04.719919 25019 leader_election.cc:302] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:04.721222 27323 raft_consensus.cc:2781] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:04.721398 27323 raft_consensus.cc:686] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:04.721560 27323 consensus_queue.cc:237] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:04.727885 22225 catalog_manager.cc:5484] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:04.744683 24932 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:04.749964 24799 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:04.752027 22634 raft_consensus.cc:1203] T f7828cb273c64934853b2cd38368402f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:04.753779 25063 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:04.757616 25195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:04.759024 22634 raft_consensus.cc:1203] T cc97164c8cfe4f91afabcc606d9821ef P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:04.761840 22499 raft_consensus.cc:1203] T af4d275b38ab49e7ac63e2fb9ffd8c07 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:04.768045 22352 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:04.790048 22483 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:04.792773 24820 raft_consensus.cc:1203] T 9f44c905bfaa4db093a1b39b6c21e03c P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:04.796422 22614 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:04.804091 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:04.845790 25084 consensus_queue.cc:237] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:04.847039 22371 raft_consensus.cc:1261] T f7828cb273c64934853b2cd38368402f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:04.847105 22634 raft_consensus.cc:1261] T f7828cb273c64934853b2cd38368402f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:04.847507 27323 consensus_queue.cc:1035] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:04.847577 22227 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:04.847769 28464 consensus_queue.cc:1035] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:04.850314 28493 raft_consensus.cc:2932] T f7828cb273c64934853b2cd38368402f P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:04.851262 22634 raft_consensus.cc:2932] T f7828cb273c64934853b2cd38368402f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:04.851708 22371 raft_consensus.cc:2932] T f7828cb273c64934853b2cd38368402f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:04.852370 24820 raft_consensus.cc:1261] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:04.852761 28464 consensus_queue.cc:1035] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:04.853588 25215 raft_consensus.cc:1261] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:04.854169 27323 consensus_queue.cc:1035] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:04.854480 22227 catalog_manager.cc:5484] T f7828cb273c64934853b2cd38368402f P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:04.858029 28301 tablet.cc:1722] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:04.858264 28500 tablet.cc:1722] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:04.858366 28301 tablet.cc:1620] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:04.858515 28500 tablet.cc:1620] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:04.857836 22230 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 4420bb923c86440b8684cc6d23d98cf6 (table rebalancer_dynamic_table [id=ef0cc0fa74c142299578e1778a636bca]). Expected version 1 got 0
I20240502 07:59:04.859578 22223 catalog_manager.cc:5576] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661) does not have the latest schema for tablet 4420bb923c86440b8684cc6d23d98cf6 (table rebalancer_dynamic_table [id=ef0cc0fa74c142299578e1778a636bca]). Expected version 1 got 0
I20240502 07:59:04.859800 24932 tablet_service.cc:1514] Processing DeleteTablet for tablet f7828cb273c64934853b2cd38368402f with delete_type TABLET_DATA_TOMBSTONED (TS 1aecef81733c47b3b468504934357231 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:04.860045 28418 tablet_replica.cc:330] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:04.860199 28418 raft_consensus.cc:2229] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:04.860352 28418 raft_consensus.cc:2258] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:04.860805 28418 ts_tablet_manager.cc:1875] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:04.863005 28418 ts_tablet_manager.cc:1888] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:04.862970 28105 consensus_queue.cc:1035] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:04.863075 28418 log.cc:1196] T f7828cb273c64934853b2cd38368402f P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/f7828cb273c64934853b2cd38368402f
I20240502 07:59:04.864079 22223 catalog_manager.cc:5576] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661) does not have the latest schema for tablet 4420bb923c86440b8684cc6d23d98cf6 (table rebalancer_dynamic_table [id=ef0cc0fa74c142299578e1778a636bca]). Expected version 1 got 0
I20240502 07:59:04.865077 28502 tablet.cc:1722] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:04.865504 28502 tablet.cc:1620] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:04.866693 22221 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=ef0cc0fa74c142299578e1778a636bca] alter complete (version 1)
I20240502 07:59:04.867581 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet f7828cb273c64934853b2cd38368402f (table rebalancer_extra_table_8 [id=72f8b22955164639b2802532262e4e2f]) successfully deleted
I20240502 07:59:04.868126 28015 consensus_queue.cc:1035] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:04.873236 28403 consensus_queue.cc:1035] T 5b952c7cbc2447ab98c17bcb2f80c8ac P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:04.879009 27752 consensus_queue.cc:1035] T ecaf2063c4cb4d2dab58354270c60cb2 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:04.892204 22499 raft_consensus.cc:1203] T 288b33b7a5f84af59bde5fae34cbe924 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:04.916564 27965 consensus_queue.cc:1035] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:04.920408 27288 consensus_queue.cc:1035] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:04.971910 22221 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:04.976663 25195 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:04.978865 28500 tablet.cc:1722] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:04.979238 28500 tablet.cc:1620] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:04.980477 24932 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:04.980535 28502 tablet.cc:1722] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:04.980878 28502 tablet.cc:1620] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:04.985304 28301 tablet.cc:1722] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:04.985643 28301 tablet.cc:1620] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:04.986677 22221 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=ef0cc0fa74c142299578e1778a636bca] alter complete (version 2)
I20240502 07:59:04.990617 24799 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:04.992345 22483 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:04.998095 22614 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:04.998617 22352 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:05.019930 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:05.028621 25063 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:05.042950 22228 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_22"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:05.052765 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 2ea24049cec547b0a6cb85be6033eb3d (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=1f887443a9c44591b3d6fac48282f904]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:05.053092 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ea24049cec547b0a6cb85be6033eb3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.053141 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 060e4ff86b7c4fb58510c4e90b0cbe36 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=1f887443a9c44591b3d6fac48282f904]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:05.053438 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 060e4ff86b7c4fb58510c4e90b0cbe36. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.053860 25195 tablet_service.cc:1467] Processing CreateTablet for tablet 2ea24049cec547b0a6cb85be6033eb3d (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=1f887443a9c44591b3d6fac48282f904]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:05.054108 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ea24049cec547b0a6cb85be6033eb3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.055644 22351 tablet_service.cc:1467] Processing CreateTablet for tablet e6cbb33529514aa0be60d0a2d70fe9a8 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=1f887443a9c44591b3d6fac48282f904]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:05.055729 24799 tablet_service.cc:1467] Processing CreateTablet for tablet 2ea24049cec547b0a6cb85be6033eb3d (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=1f887443a9c44591b3d6fac48282f904]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:05.055872 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6cbb33529514aa0be60d0a2d70fe9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.055979 24799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ea24049cec547b0a6cb85be6033eb3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.056767 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 060e4ff86b7c4fb58510c4e90b0cbe36 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=1f887443a9c44591b3d6fac48282f904]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:05.057039 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 060e4ff86b7c4fb58510c4e90b0cbe36. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.057722 24800 tablet_service.cc:1467] Processing CreateTablet for tablet e6cbb33529514aa0be60d0a2d70fe9a8 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=1f887443a9c44591b3d6fac48282f904]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:05.057942 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6cbb33529514aa0be60d0a2d70fe9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.058007 28309 tablet_bootstrap.cc:492] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:05.058873 28309 tablet_bootstrap.cc:654] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.059769 28313 tablet_bootstrap.cc:492] T e6cbb33529514aa0be60d0a2d70fe9a8 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:05.059964 22745 tablet_service.cc:1467] Processing CreateTablet for tablet e6cbb33529514aa0be60d0a2d70fe9a8 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=1f887443a9c44591b3d6fac48282f904]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:05.060184 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6cbb33529514aa0be60d0a2d70fe9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.060585 28313 tablet_bootstrap.cc:654] T e6cbb33529514aa0be60d0a2d70fe9a8 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.062959 28562 tablet_bootstrap.cc:492] T 2ea24049cec547b0a6cb85be6033eb3d P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:05.063472 28563 tablet_bootstrap.cc:492] T 060e4ff86b7c4fb58510c4e90b0cbe36 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:05.064317 28563 tablet_bootstrap.cc:654] T 060e4ff86b7c4fb58510c4e90b0cbe36 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.065232 28562 tablet_bootstrap.cc:654] T 2ea24049cec547b0a6cb85be6033eb3d P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.065359 28309 tablet_bootstrap.cc:492] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:05.065443 28309 ts_tablet_manager.cc:1367] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20240502 07:59:05.065852 28309 raft_consensus.cc:348] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:05.065950 28309 raft_consensus.cc:374] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.065977 28309 raft_consensus.cc:729] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.066082 28309 consensus_queue.cc:260] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:05.066097 28564 tablet_bootstrap.cc:492] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:05.066298 28309 ts_tablet_manager.cc:1398] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:05.066969 28564 tablet_bootstrap.cc:654] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.067659 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 060e4ff86b7c4fb58510c4e90b0cbe36 (DEFAULT_TABLE table=rebalancer_extra_table_22 [id=1f887443a9c44591b3d6fac48282f904]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:05.067895 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 060e4ff86b7c4fb58510c4e90b0cbe36. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.069481 28564 tablet_bootstrap.cc:492] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:05.069617 28564 ts_tablet_manager.cc:1367] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:05.070554 28564 raft_consensus.cc:348] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:05.070873 28564 raft_consensus.cc:374] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.070956 28564 raft_consensus.cc:729] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.071187 28562 tablet_bootstrap.cc:492] T 2ea24049cec547b0a6cb85be6033eb3d P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:05.071169 28564 consensus_queue.cc:260] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:05.071319 28562 ts_tablet_manager.cc:1367] T 2ea24049cec547b0a6cb85be6033eb3d P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:59:05.071404 28313 tablet_bootstrap.cc:492] T e6cbb33529514aa0be60d0a2d70fe9a8 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:05.071482 28563 tablet_bootstrap.cc:492] T 060e4ff86b7c4fb58510c4e90b0cbe36 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:05.071491 28313 ts_tablet_manager.cc:1367] T e6cbb33529514aa0be60d0a2d70fe9a8 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 07:59:05.071578 28563 ts_tablet_manager.cc:1367] T 060e4ff86b7c4fb58510c4e90b0cbe36 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 07:59:05.071915 28313 raft_consensus.cc:348] T e6cbb33529514aa0be60d0a2d70fe9a8 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:05.072021 28313 raft_consensus.cc:374] T e6cbb33529514aa0be60d0a2d70fe9a8 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.072057 28313 raft_consensus.cc:729] T e6cbb33529514aa0be60d0a2d70fe9a8 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.072171 28313 consensus_queue.cc:260] T e6cbb33529514aa0be60d0a2d70fe9a8 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:05.072422 28313 ts_tablet_manager.cc:1398] T e6cbb33529514aa0be60d0a2d70fe9a8 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:05.072389 28562 raft_consensus.cc:348] T 2ea24049cec547b0a6cb85be6033eb3d P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:05.072480 28562 raft_consensus.cc:374] T 2ea24049cec547b0a6cb85be6033eb3d P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.072512 28562 raft_consensus.cc:729] T 2ea24049cec547b0a6cb85be6033eb3d P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.072598 28313 tablet_bootstrap.cc:492] T 2ea24049cec547b0a6cb85be6033eb3d P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:05.072625 28562 consensus_queue.cc:260] T 2ea24049cec547b0a6cb85be6033eb3d P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:05.073051 28562 ts_tablet_manager.cc:1398] T 2ea24049cec547b0a6cb85be6033eb3d P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:05.073199 28562 tablet_bootstrap.cc:492] T e6cbb33529514aa0be60d0a2d70fe9a8 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:05.073505 28563 raft_consensus.cc:348] T 060e4ff86b7c4fb58510c4e90b0cbe36 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:05.073618 28563 raft_consensus.cc:374] T 060e4ff86b7c4fb58510c4e90b0cbe36 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.073920 28563 raft_consensus.cc:729] T 060e4ff86b7c4fb58510c4e90b0cbe36 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.074064 28562 tablet_bootstrap.cc:654] T e6cbb33529514aa0be60d0a2d70fe9a8 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.074044 28563 consensus_queue.cc:260] T 060e4ff86b7c4fb58510c4e90b0cbe36 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:05.074311 28563 ts_tablet_manager.cc:1398] T 060e4ff86b7c4fb58510c4e90b0cbe36 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:05.074478 28563 tablet_bootstrap.cc:492] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:05.075412 28563 tablet_bootstrap.cc:654] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.075755 28313 tablet_bootstrap.cc:654] T 2ea24049cec547b0a6cb85be6033eb3d P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.076817 28563 tablet_bootstrap.cc:492] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:05.077121 28563 ts_tablet_manager.cc:1367] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:05.071650 28564 ts_tablet_manager.cc:1398] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:05.077675 28563 raft_consensus.cc:348] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:05.077852 28563 raft_consensus.cc:374] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.078575 28563 raft_consensus.cc:729] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.078768 28563 consensus_queue.cc:260] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:05.079159 28563 ts_tablet_manager.cc:1398] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:05.080628 28313 tablet_bootstrap.cc:492] T 2ea24049cec547b0a6cb85be6033eb3d P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:05.080739 28313 ts_tablet_manager.cc:1367] T 2ea24049cec547b0a6cb85be6033eb3d P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:59:05.080809 28562 tablet_bootstrap.cc:492] T e6cbb33529514aa0be60d0a2d70fe9a8 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:05.080905 28562 ts_tablet_manager.cc:1367] T e6cbb33529514aa0be60d0a2d70fe9a8 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:59:05.081161 28313 raft_consensus.cc:348] T 2ea24049cec547b0a6cb85be6033eb3d P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:05.081272 28313 raft_consensus.cc:374] T 2ea24049cec547b0a6cb85be6033eb3d P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.081301 28313 raft_consensus.cc:729] T 2ea24049cec547b0a6cb85be6033eb3d P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.081442 28313 consensus_queue.cc:260] T 2ea24049cec547b0a6cb85be6033eb3d P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:05.081518 28562 raft_consensus.cc:348] T e6cbb33529514aa0be60d0a2d70fe9a8 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:05.081652 28562 raft_consensus.cc:374] T e6cbb33529514aa0be60d0a2d70fe9a8 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.081687 28562 raft_consensus.cc:729] T e6cbb33529514aa0be60d0a2d70fe9a8 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.081713 28313 ts_tablet_manager.cc:1398] T 2ea24049cec547b0a6cb85be6033eb3d P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:05.081828 28562 consensus_queue.cc:260] T e6cbb33529514aa0be60d0a2d70fe9a8 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:05.082110 28562 ts_tablet_manager.cc:1398] T e6cbb33529514aa0be60d0a2d70fe9a8 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:05.082281 28562 tablet_bootstrap.cc:492] T 060e4ff86b7c4fb58510c4e90b0cbe36 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:05.083184 28562 tablet_bootstrap.cc:654] T 060e4ff86b7c4fb58510c4e90b0cbe36 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.088127 28562 tablet_bootstrap.cc:492] T 060e4ff86b7c4fb58510c4e90b0cbe36 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:05.088536 28562 ts_tablet_manager.cc:1367] T 060e4ff86b7c4fb58510c4e90b0cbe36 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:05.088999 22228 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:05.089457 28562 raft_consensus.cc:348] T 060e4ff86b7c4fb58510c4e90b0cbe36 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:05.089696 28562 raft_consensus.cc:374] T 060e4ff86b7c4fb58510c4e90b0cbe36 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.089785 28562 raft_consensus.cc:729] T 060e4ff86b7c4fb58510c4e90b0cbe36 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.089975 28562 consensus_queue.cc:260] T 060e4ff86b7c4fb58510c4e90b0cbe36 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:05.090570 28562 ts_tablet_manager.cc:1398] T 060e4ff86b7c4fb58510c4e90b0cbe36 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:05.096124 28500 tablet.cc:1722] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:05.096817 28502 tablet.cc:1722] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:05.098124 28301 tablet.cc:1722] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:05.103525 22228 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=ef0cc0fa74c142299578e1778a636bca] alter complete (version 3)
I20240502 07:59:05.117681 25215 consensus_queue.cc:237] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:05.119223 24820 raft_consensus.cc:1261] T e3fd481f3c78497eb629b0180c07674f P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:05.119442 22634 raft_consensus.cc:1261] T e3fd481f3c78497eb629b0180c07674f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:05.119679 27752 consensus_queue.cc:1035] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:05.119957 27752 consensus_queue.cc:1035] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:05.121793 28593 raft_consensus.cc:2932] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:05.122282 24820 raft_consensus.cc:2932] T e3fd481f3c78497eb629b0180c07674f P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:05.122300 22634 raft_consensus.cc:2932] T e3fd481f3c78497eb629b0180c07674f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:05.125150 22228 catalog_manager.cc:5484] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:05.128883 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet e3fd481f3c78497eb629b0180c07674f with delete_type TABLET_DATA_TOMBSTONED (TS 227eec4b5f014d349661f6e0b39b9055 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:05.129276 28599 tablet_replica.cc:330] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:05.129631 28599 raft_consensus.cc:2229] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.129951 28599 raft_consensus.cc:2258] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.130432 28599 ts_tablet_manager.cc:1875] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.132194 28599 ts_tablet_manager.cc:1888] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:05.132266 28599 log.cc:1196] T e3fd481f3c78497eb629b0180c07674f P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/e3fd481f3c78497eb629b0180c07674f
I20240502 07:59:05.132930 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet e3fd481f3c78497eb629b0180c07674f (table rebalancer_extra_table_10 [id=03e205afe2d642629a8ba4c48516b9d5]) successfully deleted
I20240502 07:59:05.138381 28105 raft_consensus.cc:482] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:05.138506 28105 raft_consensus.cc:504] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:05.138927 28105 leader_election.cc:288] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:05.139318 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea24049cec547b0a6cb85be6033eb3d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:05.139470 22764 raft_consensus.cc:2454] T 2ea24049cec547b0a6cb85be6033eb3d P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:05.139506 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea24049cec547b0a6cb85be6033eb3d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:05.139665 24820 raft_consensus.cc:2454] T 2ea24049cec547b0a6cb85be6033eb3d P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:05.139772 25151 leader_election.cc:302] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:05.139946 28105 raft_consensus.cc:2781] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:05.140009 28105 raft_consensus.cc:482] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:05.140055 28105 raft_consensus.cc:3037] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.140992 28105 raft_consensus.cc:504] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:05.141372 28105 leader_election.cc:288] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:05.144599 28550 raft_consensus.cc:482] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:05.144709 28550 raft_consensus.cc:504] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:05.145058 28550 leader_election.cc:288] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:05.145426 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060e4ff86b7c4fb58510c4e90b0cbe36" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:05.145427 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060e4ff86b7c4fb58510c4e90b0cbe36" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:05.145558 22372 raft_consensus.cc:2454] T 060e4ff86b7c4fb58510c4e90b0cbe36 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:05.145570 22764 raft_consensus.cc:2454] T 060e4ff86b7c4fb58510c4e90b0cbe36 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:05.145589 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea24049cec547b0a6cb85be6033eb3d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:05.145697 22763 raft_consensus.cc:3037] T 2ea24049cec547b0a6cb85be6033eb3d P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.145838 22439 leader_election.cc:302] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:05.145973 28550 raft_consensus.cc:2781] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:05.146027 28550 raft_consensus.cc:482] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:05.146059 28550 raft_consensus.cc:3037] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.146790 22763 raft_consensus.cc:2454] T 2ea24049cec547b0a6cb85be6033eb3d P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:05.147092 25151 leader_election.cc:302] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:05.147208 28550 raft_consensus.cc:504] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:05.147269 28105 raft_consensus.cc:2781] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:05.147341 28105 raft_consensus.cc:686] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:05.147445 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea24049cec547b0a6cb85be6033eb3d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:05.147558 24820 raft_consensus.cc:3037] T 2ea24049cec547b0a6cb85be6033eb3d P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.147495 28105 consensus_queue.cc:237] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:05.147584 28550 leader_election.cc:288] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:05.148056 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060e4ff86b7c4fb58510c4e90b0cbe36" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:05.148175 22372 raft_consensus.cc:3037] T 060e4ff86b7c4fb58510c4e90b0cbe36 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.148591 24820 raft_consensus.cc:2454] T 2ea24049cec547b0a6cb85be6033eb3d P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:05.149361 22372 raft_consensus.cc:2454] T 060e4ff86b7c4fb58510c4e90b0cbe36 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:05.149638 22439 leader_election.cc:302] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:05.149796 28550 raft_consensus.cc:2781] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:05.149868 28550 raft_consensus.cc:686] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:59:05.149994 28550 consensus_queue.cc:237] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:05.150187 22223 catalog_manager.cc:5484] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:05.152200 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "060e4ff86b7c4fb58510c4e90b0cbe36" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:05.152333 22763 raft_consensus.cc:3037] T 060e4ff86b7c4fb58510c4e90b0cbe36 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.152288 22221 catalog_manager.cc:5484] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:05.153281 22763 raft_consensus.cc:2454] T 060e4ff86b7c4fb58510c4e90b0cbe36 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:05.205696 22227 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:05.205850 22227 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:05.208045 22227 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 4420bb923c86440b8684cc6d23d98cf6
I20240502 07:59:05.208609 24799 tablet_service.cc:1514] Processing DeleteTablet for tablet 4420bb923c86440b8684cc6d23d98cf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:05 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:59:05.209040 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet 4420bb923c86440b8684cc6d23d98cf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:05 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:05.209223 25195 tablet_service.cc:1514] Processing DeleteTablet for tablet 4420bb923c86440b8684cc6d23d98cf6 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:05 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:05.209344 28638 tablet_replica.cc:330] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:59:05.209426 28599 tablet_replica.cc:330] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:05.209524 28638 raft_consensus.cc:2229] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.209546 28599 raft_consensus.cc:2229] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:05.209722 28638 raft_consensus.cc:2258] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.209771 28599 raft_consensus.cc:2258] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.210053 28403 raft_consensus.cc:482] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:05.210151 28403 raft_consensus.cc:504] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:05.210346 28638 ts_tablet_manager.cc:1875] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:05.210430 28599 ts_tablet_manager.cc:1875] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:05.210585 28403 leader_election.cc:288] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:05.211321 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6cbb33529514aa0be60d0a2d70fe9a8" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:05.211433 24820 raft_consensus.cc:2454] T e6cbb33529514aa0be60d0a2d70fe9a8 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:05.211386 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6cbb33529514aa0be60d0a2d70fe9a8" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:05.211558 22763 raft_consensus.cc:2454] T e6cbb33529514aa0be60d0a2d70fe9a8 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:05.212018 22308 leader_election.cc:302] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:05.212527 28599 ts_tablet_manager.cc:1888] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:05.212596 28599 log.cc:1196] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/4420bb923c86440b8684cc6d23d98cf6
I20240502 07:59:05.212885 28638 ts_tablet_manager.cc:1888] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:05.212900 28599 ts_tablet_manager.cc:1909] T 4420bb923c86440b8684cc6d23d98cf6 P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:59:05.213064 28639 tablet_replica.cc:330] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:05.213214 28638 log.cc:1196] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/4420bb923c86440b8684cc6d23d98cf6
I20240502 07:59:05.213265 28639 raft_consensus.cc:2229] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.213456 28639 raft_consensus.cc:2258] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.213601 28638 ts_tablet_manager.cc:1909] T 4420bb923c86440b8684cc6d23d98cf6 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:59:05.214061 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet 4420bb923c86440b8684cc6d23d98cf6 (table rebalancer_dynamic_table_new_name [id=ef0cc0fa74c142299578e1778a636bca]) successfully deleted
I20240502 07:59:05.214107 28639 ts_tablet_manager.cc:1875] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:05.214331 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet 4420bb923c86440b8684cc6d23d98cf6 (table rebalancer_dynamic_table_new_name [id=ef0cc0fa74c142299578e1778a636bca]) successfully deleted
I20240502 07:59:05.214493 28015 raft_consensus.cc:2781] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:05.214553 28015 raft_consensus.cc:482] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:05.214601 28015 raft_consensus.cc:3037] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.215597 28015 raft_consensus.cc:504] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:05.215945 28015 leader_election.cc:288] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:05.216516 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6cbb33529514aa0be60d0a2d70fe9a8" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:05.216540 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6cbb33529514aa0be60d0a2d70fe9a8" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:05.216562 28639 ts_tablet_manager.cc:1888] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:05.216626 28639 log.cc:1196] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/4420bb923c86440b8684cc6d23d98cf6
I20240502 07:59:05.216638 24820 raft_consensus.cc:3037] T e6cbb33529514aa0be60d0a2d70fe9a8 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.216639 22763 raft_consensus.cc:3037] T e6cbb33529514aa0be60d0a2d70fe9a8 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.216895 28639 ts_tablet_manager.cc:1909] T 4420bb923c86440b8684cc6d23d98cf6 P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:59:05.217525 24820 raft_consensus.cc:2454] T e6cbb33529514aa0be60d0a2d70fe9a8 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:05.217624 22763 raft_consensus.cc:2454] T e6cbb33529514aa0be60d0a2d70fe9a8 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:05.217864 22307 leader_election.cc:302] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:05.218012 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 4420bb923c86440b8684cc6d23d98cf6 (table rebalancer_dynamic_table_new_name [id=ef0cc0fa74c142299578e1778a636bca]) successfully deleted
I20240502 07:59:05.218170 28015 raft_consensus.cc:2781] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:05.218266 28015 raft_consensus.cc:686] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:59:05.218405 28015 consensus_queue.cc:237] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:05.220826 22224 catalog_manager.cc:5484] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:05.245877 24932 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:05.246938 25063 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:05.252954 24799 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:05.253928 22351 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:05.260049 25195 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:05.270268 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:05.291857 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:05.296661 22614 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:05.310173 22224 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:05.315490 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 92553344358b44e19998725af84ba8cf (DEFAULT_TABLE table=rebalancer_dynamic_table [id=ff58e3e58b244b41b7031f5ddb84caa1]), partition=
I20240502 07:59:05.315526 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 92553344358b44e19998725af84ba8cf (DEFAULT_TABLE table=rebalancer_dynamic_table [id=ff58e3e58b244b41b7031f5ddb84caa1]), partition=
I20240502 07:59:05.315759 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92553344358b44e19998725af84ba8cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.315778 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92553344358b44e19998725af84ba8cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.317425 28564 tablet_bootstrap.cc:492] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:05.317607 25195 tablet_service.cc:1467] Processing CreateTablet for tablet 92553344358b44e19998725af84ba8cf (DEFAULT_TABLE table=rebalancer_dynamic_table [id=ff58e3e58b244b41b7031f5ddb84caa1]), partition=
I20240502 07:59:05.317836 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92553344358b44e19998725af84ba8cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.318328 28564 tablet_bootstrap.cc:654] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.319458 28564 tablet_bootstrap.cc:492] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:05.319550 28564 ts_tablet_manager.cc:1367] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:05.320091 28309 tablet_bootstrap.cc:492] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:05.320040 28564 raft_consensus.cc:348] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:05.320150 28564 raft_consensus.cc:374] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.320189 28564 raft_consensus.cc:729] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.320319 28564 consensus_queue.cc:260] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:05.320636 28564 ts_tablet_manager.cc:1398] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:05.321017 28309 tablet_bootstrap.cc:654] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.321280 28652 tablet_bootstrap.cc:492] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:05.322314 28652 tablet_bootstrap.cc:654] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.323354 28652 tablet_bootstrap.cc:492] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:05.323449 28652 ts_tablet_manager.cc:1367] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:05.323874 28652 raft_consensus.cc:348] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:05.323984 28652 raft_consensus.cc:374] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.324012 28652 raft_consensus.cc:729] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.324133 28652 consensus_queue.cc:260] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:05.324412 28652 ts_tablet_manager.cc:1398] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:05.324556 28309 tablet_bootstrap.cc:492] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:05.324642 28309 ts_tablet_manager.cc:1367] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:05.325105 28309 raft_consensus.cc:348] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:05.325206 28309 raft_consensus.cc:374] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.325234 28309 raft_consensus.cc:729] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.325389 28309 consensus_queue.cc:260] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:05.325675 28309 ts_tablet_manager.cc:1398] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:05.351608 22634 consensus_queue.cc:237] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:05.352792 24820 raft_consensus.cc:1261] T b270d89546214f4d9d15193f4f8e04d5 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.352795 22763 raft_consensus.cc:1261] T b270d89546214f4d9d15193f4f8e04d5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.353197 27288 consensus_queue.cc:1035] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.353408 27813 consensus_queue.cc:1035] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.355152 27965 raft_consensus.cc:2932] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:05.355578 24820 raft_consensus.cc:2932] T b270d89546214f4d9d15193f4f8e04d5 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:05.355708 22763 raft_consensus.cc:2932] T b270d89546214f4d9d15193f4f8e04d5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:05.358461 22226 catalog_manager.cc:5484] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:05.361451 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet b270d89546214f4d9d15193f4f8e04d5 with delete_type TABLET_DATA_TOMBSTONED (TS fabddb16c85944648a710ec52fc7244c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:05.362360 28678 tablet_replica.cc:330] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:05.362553 28678 raft_consensus.cc:2229] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.362699 28678 raft_consensus.cc:2258] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.363193 28678 ts_tablet_manager.cc:1875] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.365164 28678 ts_tablet_manager.cc:1888] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:05.365252 28678 log.cc:1196] T b270d89546214f4d9d15193f4f8e04d5 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/b270d89546214f4d9d15193f4f8e04d5
I20240502 07:59:05.365783 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet b270d89546214f4d9d15193f4f8e04d5 (table rebalance_test_table_17 [id=1ec511fc66814cefa73a9b7acff4eea9]) successfully deleted
I20240502 07:59:05.365947 28008 raft_consensus.cc:482] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:05.366050 28008 raft_consensus.cc:504] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:05.366477 28008 leader_election.cc:288] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:05.366730 22499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92553344358b44e19998725af84ba8cf" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:05.366843 22499 raft_consensus.cc:2454] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:05.366955 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92553344358b44e19998725af84ba8cf" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:05.367084 25215 raft_consensus.cc:2454] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:05.367121 24885 leader_election.cc:302] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:05.367362 28008 raft_consensus.cc:2781] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:05.367429 28008 raft_consensus.cc:482] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:05.367489 28008 raft_consensus.cc:3037] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.368481 28008 raft_consensus.cc:504] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:05.368809 28008 leader_election.cc:288] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:05.369105 22499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92553344358b44e19998725af84ba8cf" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:05.369207 22499 raft_consensus.cc:3037] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.369568 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92553344358b44e19998725af84ba8cf" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:05.369671 25215 raft_consensus.cc:3037] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.370582 25215 raft_consensus.cc:2454] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:05.370843 22499 raft_consensus.cc:2454] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:05.371137 24885 leader_election.cc:302] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:05.371310 28008 raft_consensus.cc:2781] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:05.371436 28008 raft_consensus.cc:686] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:05.371582 28008 consensus_queue.cc:237] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:05.373950 22224 catalog_manager.cc:5484] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:05.455018 24932 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:05.455906 22351 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:05.463649 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:05.469802 24799 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:05.471598 22614 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:05.473788 25195 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:05.478019 22230 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:05.494210 25063 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:05.504287 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:05.516129 22499 raft_consensus.cc:1261] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:05.516595 28686 consensus_queue.cc:1035] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:05.519827 22230 catalog_manager.cc:5576] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625) does not have the latest schema for tablet 92553344358b44e19998725af84ba8cf (table rebalancer_dynamic_table [id=ff58e3e58b244b41b7031f5ddb84caa1]). Expected version 1 got 0
I20240502 07:59:05.520666 25214 raft_consensus.cc:1261] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:05.521257 28686 consensus_queue.cc:1035] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:05.525116 22230 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 92553344358b44e19998725af84ba8cf (table rebalancer_dynamic_table [id=ff58e3e58b244b41b7031f5ddb84caa1]). Expected version 1 got 0
I20240502 07:59:05.534013 28301 tablet.cc:1722] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:05.534372 28301 tablet.cc:1620] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:05.538360 22230 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=ff58e3e58b244b41b7031f5ddb84caa1] alter complete (version 1)
I20240502 07:59:05.538539 28734 tablet.cc:1722] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:05.538836 28734 tablet.cc:1620] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c: MemRowSet was empty: no flush needed.
I20240502 07:59:05.539822 22230 catalog_manager.cc:5576] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625) does not have the latest schema for tablet 92553344358b44e19998725af84ba8cf (table rebalancer_dynamic_table [id=ff58e3e58b244b41b7031f5ddb84caa1]). Expected version 1 got 0
I20240502 07:59:05.543509 28733 tablet.cc:1722] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:05.543833 28733 tablet.cc:1620] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:05.556939 27888 consensus_queue.cc:1035] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:05.561167 27888 consensus_queue.cc:1035] T 060e4ff86b7c4fb58510c4e90b0cbe36 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:05.576591 22499 consensus_queue.cc:237] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:05.577845 25214 raft_consensus.cc:1261] T dbf60f171dfe4c88840587bf0df64cac P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.578030 22372 raft_consensus.cc:1261] T dbf60f171dfe4c88840587bf0df64cac P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.578235 28550 consensus_queue.cc:1035] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.578404 27888 consensus_queue.cc:1035] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.580266 27888 raft_consensus.cc:2932] T dbf60f171dfe4c88840587bf0df64cac P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:05.580657 22372 raft_consensus.cc:2932] T dbf60f171dfe4c88840587bf0df64cac P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:05.580664 25215 raft_consensus.cc:2932] T dbf60f171dfe4c88840587bf0df64cac P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:05.583391 22224 catalog_manager.cc:5484] T dbf60f171dfe4c88840587bf0df64cac P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:05.586503 22614 tablet_service.cc:1514] Processing DeleteTablet for tablet dbf60f171dfe4c88840587bf0df64cac with delete_type TABLET_DATA_TOMBSTONED (TS e51b10af49484aaa8f0267aebc1bf27f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:05.587852 28758 tablet_replica.cc:330] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:05.588070 28758 raft_consensus.cc:2229] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.588299 28758 raft_consensus.cc:2258] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.589088 28758 ts_tablet_manager.cc:1875] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.589671 28105 consensus_queue.cc:1035] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:05.591792 28758 ts_tablet_manager.cc:1888] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:05.591876 28758 log.cc:1196] T dbf60f171dfe4c88840587bf0df64cac P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/dbf60f171dfe4c88840587bf0df64cac
I20240502 07:59:05.592479 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet dbf60f171dfe4c88840587bf0df64cac (table rebalance_test_table_5 [id=80b9c0f932754699a4f41d2c2188b70b]) successfully deleted
I20240502 07:59:05.596374 28593 consensus_queue.cc:1035] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:05.643057 22221 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:05.650826 28733 tablet.cc:1722] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:05.651176 28733 tablet.cc:1620] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:05.652129 28301 tablet.cc:1722] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:05.652427 28301 tablet.cc:1620] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:05.655936 28734 tablet.cc:1722] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:05.656250 28734 tablet.cc:1620] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c: MemRowSet was empty: no flush needed.
I20240502 07:59:05.657369 22230 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=ff58e3e58b244b41b7031f5ddb84caa1] alter complete (version 2)
I20240502 07:59:05.695220 25195 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:05.705792 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:05.706033 24799 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:05.706918 22351 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:05.707331 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:05.707597 24932 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:05.708045 22614 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:05.712721 25063 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:05.760673 22221 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:05.768252 28733 tablet.cc:1722] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:05.770874 22229 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_24"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:05.770996 28734 tablet.cc:1722] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:05.777477 22224 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=ff58e3e58b244b41b7031f5ddb84caa1] alter complete (version 3)
I20240502 07:59:05.777243 28301 tablet.cc:1722] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:05.781947 24799 tablet_service.cc:1467] Processing CreateTablet for tablet f033e8bbac8445f3af64faa67c818987 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=5489b85f07d04460928611600e2b5813]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:05.782037 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 5034380b5c0846faa1494140d67203cc (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=5489b85f07d04460928611600e2b5813]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:05.782289 24799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f033e8bbac8445f3af64faa67c818987. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.782837 25062 tablet_service.cc:1467] Processing CreateTablet for tablet 5034380b5c0846faa1494140d67203cc (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=5489b85f07d04460928611600e2b5813]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:05.782980 25063 tablet_service.cc:1467] Processing CreateTablet for tablet cc10e78159fd4892a746b28861515b72 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=5489b85f07d04460928611600e2b5813]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:05.783013 22614 tablet_service.cc:1467] Processing CreateTablet for tablet f033e8bbac8445f3af64faa67c818987 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=5489b85f07d04460928611600e2b5813]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:05.783114 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5034380b5c0846faa1494140d67203cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.783260 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f033e8bbac8445f3af64faa67c818987. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.783485 24930 tablet_service.cc:1467] Processing CreateTablet for tablet 5034380b5c0846faa1494140d67203cc (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=5489b85f07d04460928611600e2b5813]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:05.783749 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5034380b5c0846faa1494140d67203cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.784790 24932 tablet_service.cc:1467] Processing CreateTablet for tablet cc10e78159fd4892a746b28861515b72 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=5489b85f07d04460928611600e2b5813]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:05.785006 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc10e78159fd4892a746b28861515b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.786041 28652 tablet_bootstrap.cc:492] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:05.786569 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5034380b5c0846faa1494140d67203cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.786783 25195 tablet_service.cc:1467] Processing CreateTablet for tablet cc10e78159fd4892a746b28861515b72 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=5489b85f07d04460928611600e2b5813]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:05.786974 28652 tablet_bootstrap.cc:654] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.787024 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc10e78159fd4892a746b28861515b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.787114 24931 tablet_service.cc:1467] Processing CreateTablet for tablet f033e8bbac8445f3af64faa67c818987 (DEFAULT_TABLE table=rebalancer_extra_table_24 [id=5489b85f07d04460928611600e2b5813]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:05.787309 24931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f033e8bbac8445f3af64faa67c818987. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.789175 28403 consensus_queue.cc:1035] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:05.789402 28309 tablet_bootstrap.cc:492] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:05.790331 28309 tablet_bootstrap.cc:654] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.790331 28822 tablet_bootstrap.cc:492] T f033e8bbac8445f3af64faa67c818987 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:05.790972 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc10e78159fd4892a746b28861515b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:05.791368 28822 tablet_bootstrap.cc:654] T f033e8bbac8445f3af64faa67c818987 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.792652 28652 tablet_bootstrap.cc:492] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:05.792760 28652 ts_tablet_manager.cc:1367] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:05.792770 28822 tablet_bootstrap.cc:492] T f033e8bbac8445f3af64faa67c818987 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:05.792848 28822 ts_tablet_manager.cc:1367] T f033e8bbac8445f3af64faa67c818987 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:05.793349 28822 raft_consensus.cc:348] T f033e8bbac8445f3af64faa67c818987 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:05.793462 28822 raft_consensus.cc:374] T f033e8bbac8445f3af64faa67c818987 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.793495 28822 raft_consensus.cc:729] T f033e8bbac8445f3af64faa67c818987 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.793501 28309 tablet_bootstrap.cc:492] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:05.793259 28652 raft_consensus.cc:348] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.793565 28652 raft_consensus.cc:374] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.793601 28309 ts_tablet_manager.cc:1367] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:05.793607 28652 raft_consensus.cc:729] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.793620 28822 consensus_queue.cc:260] T f033e8bbac8445f3af64faa67c818987 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:05.793733 28652 consensus_queue.cc:260] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.793910 28822 ts_tablet_manager.cc:1398] T f033e8bbac8445f3af64faa67c818987 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:05.794062 28822 tablet_bootstrap.cc:492] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:05.794071 28652 ts_tablet_manager.cc:1398] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:05.794034 28309 raft_consensus.cc:348] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.794147 28309 raft_consensus.cc:374] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.794178 28309 raft_consensus.cc:729] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.794250 28652 tablet_bootstrap.cc:492] T cc10e78159fd4892a746b28861515b72 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:05.794301 28309 consensus_queue.cc:260] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.794601 28309 ts_tablet_manager.cc:1398] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:05.795099 28823 tablet_bootstrap.cc:492] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:05.795219 28652 tablet_bootstrap.cc:654] T cc10e78159fd4892a746b28861515b72 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.796124 28822 tablet_bootstrap.cc:654] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.796450 28824 tablet_bootstrap.cc:492] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:05.797772 28824 tablet_bootstrap.cc:654] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.798712 28822 tablet_bootstrap.cc:492] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:05.798803 28822 ts_tablet_manager.cc:1367] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:05.799219 28823 tablet_bootstrap.cc:654] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.799249 28822 raft_consensus.cc:348] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.799731 28822 raft_consensus.cc:374] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.799785 28822 raft_consensus.cc:729] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.799930 28822 consensus_queue.cc:260] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.800238 28822 ts_tablet_manager.cc:1398] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:05.802264 28824 tablet_bootstrap.cc:492] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:05.802441 28824 ts_tablet_manager.cc:1367] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:05.803807 28824 raft_consensus.cc:348] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:05.803998 28824 raft_consensus.cc:374] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.804073 28824 raft_consensus.cc:729] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.804080 28652 tablet_bootstrap.cc:492] T cc10e78159fd4892a746b28861515b72 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:05.804255 28824 consensus_queue.cc:260] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:05.804719 28824 ts_tablet_manager.cc:1398] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:05.806274 28823 tablet_bootstrap.cc:492] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:05.806449 28823 ts_tablet_manager.cc:1367] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20240502 07:59:05.806624 28834 consensus_queue.cc:1035] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:05.806862 28652 ts_tablet_manager.cc:1367] T cc10e78159fd4892a746b28861515b72 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 07:59:05.807463 28652 raft_consensus.cc:348] T cc10e78159fd4892a746b28861515b72 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.807590 28652 raft_consensus.cc:374] T cc10e78159fd4892a746b28861515b72 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.807632 28652 raft_consensus.cc:729] T cc10e78159fd4892a746b28861515b72 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.808130 28823 raft_consensus.cc:348] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.808434 28823 raft_consensus.cc:374] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.808514 28823 raft_consensus.cc:729] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.808714 28823 consensus_queue.cc:260] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.807751 28652 consensus_queue.cc:260] T cc10e78159fd4892a746b28861515b72 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.810686 28652 ts_tablet_manager.cc:1398] T cc10e78159fd4892a746b28861515b72 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:05.811118 28652 tablet_bootstrap.cc:492] T f033e8bbac8445f3af64faa67c818987 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:05.812243 28652 tablet_bootstrap.cc:654] T f033e8bbac8445f3af64faa67c818987 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.812824 28823 ts_tablet_manager.cc:1398] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 07:59:05.813162 28823 tablet_bootstrap.cc:492] T cc10e78159fd4892a746b28861515b72 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:05.815147 28823 tablet_bootstrap.cc:654] T cc10e78159fd4892a746b28861515b72 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:05.817078 28652 tablet_bootstrap.cc:492] T f033e8bbac8445f3af64faa67c818987 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:05.817171 28652 ts_tablet_manager.cc:1367] T f033e8bbac8445f3af64faa67c818987 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:05.817433 28823 tablet_bootstrap.cc:492] T cc10e78159fd4892a746b28861515b72 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:05.817528 28823 ts_tablet_manager.cc:1367] T cc10e78159fd4892a746b28861515b72 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:05.817734 28652 raft_consensus.cc:348] T f033e8bbac8445f3af64faa67c818987 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:05.817826 28652 raft_consensus.cc:374] T f033e8bbac8445f3af64faa67c818987 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.817859 28652 raft_consensus.cc:729] T f033e8bbac8445f3af64faa67c818987 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.817984 28652 consensus_queue.cc:260] T f033e8bbac8445f3af64faa67c818987 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:05.817983 28823 raft_consensus.cc:348] T cc10e78159fd4892a746b28861515b72 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.818091 28823 raft_consensus.cc:374] T cc10e78159fd4892a746b28861515b72 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:05.818125 28823 raft_consensus.cc:729] T cc10e78159fd4892a746b28861515b72 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:05.818265 28652 ts_tablet_manager.cc:1398] T f033e8bbac8445f3af64faa67c818987 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:05.818243 28823 consensus_queue.cc:260] T cc10e78159fd4892a746b28861515b72 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.818526 28823 ts_tablet_manager.cc:1398] T cc10e78159fd4892a746b28861515b72 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:05.826812 22634 consensus_queue.cc:237] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.828074 24952 raft_consensus.cc:1261] T b7cdcee32c8f4cd3af30c402b8da2484 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.828182 22372 raft_consensus.cc:1261] T b7cdcee32c8f4cd3af30c402b8da2484 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:05.828457 27965 consensus_queue.cc:1035] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.829026 27965 consensus_queue.cc:1035] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:05.830050 28570 raft_consensus.cc:482] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:05.830165 28570 raft_consensus.cc:504] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.830348 27813 raft_consensus.cc:2932] T b7cdcee32c8f4cd3af30c402b8da2484 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:05.830621 28570 leader_election.cc:288] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:05.831104 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5034380b5c0846faa1494140d67203cc" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:05.831226 24952 raft_consensus.cc:2454] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:05.831508 24757 leader_election.cc:302] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:05.831709 28570 raft_consensus.cc:2781] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:05.831758 28570 raft_consensus.cc:482] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:05.831792 28570 raft_consensus.cc:3037] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.831614 22372 raft_consensus.cc:2932] T b7cdcee32c8f4cd3af30c402b8da2484 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:05.832309 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5034380b5c0846faa1494140d67203cc" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:05.832406 25084 raft_consensus.cc:2454] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:05.832720 28570 raft_consensus.cc:504] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.832887 24952 raft_consensus.cc:2932] T b7cdcee32c8f4cd3af30c402b8da2484 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:05.833050 28570 leader_election.cc:288] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:05.834684 22221 catalog_manager.cc:5484] T b7cdcee32c8f4cd3af30c402b8da2484 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:05.834826 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5034380b5c0846faa1494140d67203cc" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:05.834928 24951 raft_consensus.cc:3037] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.835836 24951 raft_consensus.cc:2454] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:05.836320 24757 leader_election.cc:302] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:05.836477 28570 raft_consensus.cc:2781] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:05.836652 28570 raft_consensus.cc:686] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:59:05.836791 28570 consensus_queue.cc:237] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.837080 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5034380b5c0846faa1494140d67203cc" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:05.837203 25084 raft_consensus.cc:3037] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.838158 25084 raft_consensus.cc:2454] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:05.839548 22229 catalog_manager.cc:5484] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:05.840865 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet b7cdcee32c8f4cd3af30c402b8da2484 with delete_type TABLET_DATA_TOMBSTONED (TS fabddb16c85944648a710ec52fc7244c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:05.841114 28678 tablet_replica.cc:330] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:05.841261 28678 raft_consensus.cc:2229] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.841451 28678 raft_consensus.cc:2258] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.842016 28678 ts_tablet_manager.cc:1875] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:05.844000 28678 ts_tablet_manager.cc:1888] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:05.844050 28678 log.cc:1196] T b7cdcee32c8f4cd3af30c402b8da2484 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/b7cdcee32c8f4cd3af30c402b8da2484
I20240502 07:59:05.844476 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet b7cdcee32c8f4cd3af30c402b8da2484 (table rebalance_test_table_9 [id=9b69b771dcfb4a00a937ed9203245903]) successfully deleted
I20240502 07:59:05.848996 28105 raft_consensus.cc:482] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:05.849088 28105 raft_consensus.cc:504] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.849550 28105 leader_election.cc:288] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:05.849891 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc10e78159fd4892a746b28861515b72" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:05.849954 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc10e78159fd4892a746b28861515b72" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:05.850030 24951 raft_consensus.cc:2454] T cc10e78159fd4892a746b28861515b72 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:05.850075 25084 raft_consensus.cc:2454] T cc10e78159fd4892a746b28861515b72 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:05.850324 25148 leader_election.cc:302] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:05.850517 28105 raft_consensus.cc:2781] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:05.850579 28105 raft_consensus.cc:482] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:05.850625 28105 raft_consensus.cc:3037] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.851428 28105 raft_consensus.cc:504] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.851739 28105 leader_election.cc:288] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:05.852084 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc10e78159fd4892a746b28861515b72" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:05.852205 24951 raft_consensus.cc:3037] T cc10e78159fd4892a746b28861515b72 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.852315 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc10e78159fd4892a746b28861515b72" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:05.852419 25084 raft_consensus.cc:3037] T cc10e78159fd4892a746b28861515b72 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.852962 24951 raft_consensus.cc:2454] T cc10e78159fd4892a746b28861515b72 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:05.853189 25084 raft_consensus.cc:2454] T cc10e78159fd4892a746b28861515b72 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:05.853236 25151 leader_election.cc:302] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:05.853389 28105 raft_consensus.cc:2781] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:05.853492 28105 raft_consensus.cc:686] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:05.853670 28105 consensus_queue.cc:237] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:05.855715 22229 catalog_manager.cc:5484] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:05.890640 22221 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:05.890795 22221 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:05.897871 22221 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 92553344358b44e19998725af84ba8cf
I20240502 07:59:05.898715 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet 92553344358b44e19998725af84ba8cf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:05 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:05.898737 24931 tablet_service.cc:1514] Processing DeleteTablet for tablet 92553344358b44e19998725af84ba8cf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:05 UTC) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:05.899006 28678 tablet_replica.cc:330] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:05.899160 28678 raft_consensus.cc:2229] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.899298 28678 raft_consensus.cc:2258] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.899633 25195 tablet_service.cc:1514] Processing DeleteTablet for tablet 92553344358b44e19998725af84ba8cf with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:05 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:05.899842 28678 ts_tablet_manager.cc:1875] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:05.899976 28896 tablet_replica.cc:330] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:05.900096 28896 raft_consensus.cc:2229] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:05.900215 28896 raft_consensus.cc:2258] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.900774 28896 ts_tablet_manager.cc:1875] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:05.902566 28678 ts_tablet_manager.cc:1888] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:05.902700 28678 log.cc:1196] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/92553344358b44e19998725af84ba8cf
I20240502 07:59:05.904270 28678 ts_tablet_manager.cc:1909] T 92553344358b44e19998725af84ba8cf P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:59:05.905143 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 92553344358b44e19998725af84ba8cf (table rebalancer_dynamic_table_new_name [id=ff58e3e58b244b41b7031f5ddb84caa1]) successfully deleted
I20240502 07:59:05.910959 28896 ts_tablet_manager.cc:1888] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:05.911067 28896 log.cc:1196] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/92553344358b44e19998725af84ba8cf
I20240502 07:59:05.914659 28895 tablet_replica.cc:330] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:05.914825 28895 raft_consensus.cc:2229] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:05.915023 28895 raft_consensus.cc:2258] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:05.916211 28895 ts_tablet_manager.cc:1875] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:05.919133 28896 ts_tablet_manager.cc:1909] T 92553344358b44e19998725af84ba8cf P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:59:05.919834 28895 ts_tablet_manager.cc:1888] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:05.919906 28895 log.cc:1196] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/92553344358b44e19998725af84ba8cf
I20240502 07:59:05.920156 28895 ts_tablet_manager.cc:1909] T 92553344358b44e19998725af84ba8cf P 1aecef81733c47b3b468504934357231: Deleting consensus metadata
I20240502 07:59:05.920248 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 92553344358b44e19998725af84ba8cf (table rebalancer_dynamic_table_new_name [id=ff58e3e58b244b41b7031f5ddb84caa1]) successfully deleted
I20240502 07:59:05.925968 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet 92553344358b44e19998725af84ba8cf (table rebalancer_dynamic_table_new_name [id=ff58e3e58b244b41b7031f5ddb84caa1]) successfully deleted
I20240502 07:59:05.940177 27813 raft_consensus.cc:482] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:05.940304 27813 raft_consensus.cc:504] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:05.940719 27813 leader_election.cc:288] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:05.940997 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f033e8bbac8445f3af64faa67c818987" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:05.941143 24820 raft_consensus.cc:2454] T f033e8bbac8445f3af64faa67c818987 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:05.941124 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f033e8bbac8445f3af64faa67c818987" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:05.941257 24951 raft_consensus.cc:2454] T f033e8bbac8445f3af64faa67c818987 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:05.941537 22569 leader_election.cc:302] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:05.941745 27813 raft_consensus.cc:2781] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:05.941808 27813 raft_consensus.cc:482] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:05.941841 27813 raft_consensus.cc:3037] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.942720 27813 raft_consensus.cc:504] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:05.943044 27813 leader_election.cc:288] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:05.943388 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f033e8bbac8445f3af64faa67c818987" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:05.943434 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f033e8bbac8445f3af64faa67c818987" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:05.943513 24820 raft_consensus.cc:3037] T f033e8bbac8445f3af64faa67c818987 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.943547 24951 raft_consensus.cc:3037] T f033e8bbac8445f3af64faa67c818987 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:05.944491 24951 raft_consensus.cc:2454] T f033e8bbac8445f3af64faa67c818987 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:05.944492 24820 raft_consensus.cc:2454] T f033e8bbac8445f3af64faa67c818987 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:05.944759 22570 leader_election.cc:302] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:05.944952 27813 raft_consensus.cc:2781] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:05.945045 27813 raft_consensus.cc:686] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:59:05.945197 27813 consensus_queue.cc:237] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:05.947244 22228 catalog_manager.cc:5484] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:05.965998 24800 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:05.969874 25063 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:05.971052 24931 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:05.971978 25195 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:05.974946 22614 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:05.976300 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:05.986670 22351 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:06.003038 22228 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:06.012549 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 5f78e3e5cb144f8783c7cbb344375f01 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=75970b92fa1644ec9b8346a7970a2e7d]), partition=
I20240502 07:59:06.012545 24931 tablet_service.cc:1467] Processing CreateTablet for tablet 5f78e3e5cb144f8783c7cbb344375f01 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=75970b92fa1644ec9b8346a7970a2e7d]), partition=
I20240502 07:59:06.012813 24931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f78e3e5cb144f8783c7cbb344375f01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.012821 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f78e3e5cb144f8783c7cbb344375f01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.013578 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 5f78e3e5cb144f8783c7cbb344375f01 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=75970b92fa1644ec9b8346a7970a2e7d]), partition=
I20240502 07:59:06.013808 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f78e3e5cb144f8783c7cbb344375f01. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.015182 28822 tablet_bootstrap.cc:492] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:06.015789 28823 tablet_bootstrap.cc:492] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:06.016049 28822 tablet_bootstrap.cc:654] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:06.016054 28652 tablet_bootstrap.cc:492] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:06.017042 28652 tablet_bootstrap.cc:654] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:06.017292 28823 tablet_bootstrap.cc:654] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:06.019390 28652 tablet_bootstrap.cc:492] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:06.019490 28652 ts_tablet_manager.cc:1367] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 07:59:06.019925 28652 raft_consensus.cc:348] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:06.020077 28652 raft_consensus.cc:374] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:06.020107 28652 raft_consensus.cc:729] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:06.020218 28652 consensus_queue.cc:260] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:06.020493 28652 ts_tablet_manager.cc:1398] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:06.023720 28822 tablet_bootstrap.cc:492] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:06.023815 28822 ts_tablet_manager.cc:1367] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 07:59:06.024236 28822 raft_consensus.cc:348] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:06.024381 28822 raft_consensus.cc:374] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:06.024427 28822 raft_consensus.cc:729] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:06.024544 28822 consensus_queue.cc:260] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:06.024821 28822 ts_tablet_manager.cc:1398] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:06.030831 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:06.038427 28823 tablet_bootstrap.cc:492] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:06.038539 28823 ts_tablet_manager.cc:1367] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.002s
I20240502 07:59:06.039045 28823 raft_consensus.cc:348] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:06.039170 28823 raft_consensus.cc:374] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:06.039206 28823 raft_consensus.cc:729] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:06.039337 28823 consensus_queue.cc:260] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:06.039623 28823 ts_tablet_manager.cc:1398] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:06.065119 22634 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c5841aadb46447e685af47b70d0feff4"
dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:47624
I20240502 07:59:06.065279 22634 raft_consensus.cc:595] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:06.244468 28570 consensus_queue.cc:1035] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:06.247275 28851 consensus_queue.cc:1035] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:06.306496 28493 raft_consensus.cc:482] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:06.306625 28493 raft_consensus.cc:504] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:06.307018 28493 leader_election.cc:288] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:06.307339 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f78e3e5cb144f8783c7cbb344375f01" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:06.307391 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f78e3e5cb144f8783c7cbb344375f01" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:06.307497 24820 raft_consensus.cc:2454] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:06.307572 24952 raft_consensus.cc:2454] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:06.307767 25019 leader_election.cc:302] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:06.307925 28493 raft_consensus.cc:2781] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:06.308012 28493 raft_consensus.cc:482] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:06.308061 28493 raft_consensus.cc:3037] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:06.308988 28493 raft_consensus.cc:504] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:06.309299 28493 leader_election.cc:288] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:06.309659 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f78e3e5cb144f8783c7cbb344375f01" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:06.309769 24820 raft_consensus.cc:3037] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:06.309780 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f78e3e5cb144f8783c7cbb344375f01" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:06.309892 24952 raft_consensus.cc:3037] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:06.310564 24820 raft_consensus.cc:2454] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:06.310627 24952 raft_consensus.cc:2454] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:06.310853 25019 leader_election.cc:302] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:06.311067 28493 raft_consensus.cc:2781] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:06.311148 28493 raft_consensus.cc:686] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:06.311312 28493 consensus_queue.cc:237] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:06.313045 22228 catalog_manager.cc:5484] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:06.338080 24800 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:06.338485 24931 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:06.338521 25195 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:06.339253 25063 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:06.341862 22351 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:06.357479 22614 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:06.365664 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:06.367842 27813 raft_consensus.cc:979] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f: : Instructing follower 1aecef81733c47b3b468504934357231 to start an election
I20240502 07:59:06.367961 27813 raft_consensus.cc:1067] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Signalling peer 1aecef81733c47b3b468504934357231 to start an election
I20240502 07:59:06.368625 24952 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c5841aadb46447e685af47b70d0feff4"
dest_uuid: "1aecef81733c47b3b468504934357231"
from {username='slave'} at 127.15.129.131:57117
I20240502 07:59:06.368794 24952 raft_consensus.cc:482] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:06.368862 24952 raft_consensus.cc:3037] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:06.369863 24952 raft_consensus.cc:504] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:06.370321 24952 leader_election.cc:288] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 2 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:06.370712 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5841aadb46447e685af47b70d0feff4" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:06.370836 22764 raft_consensus.cc:3037] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:06.371819 22764 raft_consensus.cc:2454] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 2.
I20240502 07:59:06.373571 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5841aadb46447e685af47b70d0feff4" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:06.373700 22634 raft_consensus.cc:3032] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:06.373739 22634 raft_consensus.cc:729] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:59:06.373895 22634 consensus_queue.cc:260] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:06.374106 22634 raft_consensus.cc:3037] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:06.373862 22499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5841aadb46447e685af47b70d0feff4" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:06.374264 22499 raft_consensus.cc:3037] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:06.375205 22634 raft_consensus.cc:2454] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 2.
I20240502 07:59:06.375470 22499 raft_consensus.cc:2454] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 2.
I20240502 07:59:06.376650 24885 leader_election.cc:302] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:06.376904 28686 raft_consensus.cc:2781] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:06.376979 28686 raft_consensus.cc:686] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [term 2 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:06.377149 28686 consensus_queue.cc:237] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:06.379846 22228 catalog_manager.cc:5484] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 1 to 2, leader changed from e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 2 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:06.392777 28897 consensus_queue.cc:1035] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:06.393906 28593 consensus_queue.cc:1035] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:06.395493 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:06.397886 28593 consensus_queue.cc:1035] T cc10e78159fd4892a746b28861515b72 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:06.401805 27813 consensus_queue.cc:1035] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:06.420454 22228 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:06.425311 24951 raft_consensus.cc:1261] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:06.425570 24820 raft_consensus.cc:1261] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:06.425926 28493 consensus_queue.cc:1035] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:06.426060 28464 consensus_queue.cc:1035] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:06.430871 22228 catalog_manager.cc:5576] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995) does not have the latest schema for tablet 5f78e3e5cb144f8783c7cbb344375f01 (table rebalancer_dynamic_table [id=75970b92fa1644ec9b8346a7970a2e7d]). Expected version 1 got 0
I20240502 07:59:06.433662 22228 catalog_manager.cc:5576] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661) does not have the latest schema for tablet 5f78e3e5cb144f8783c7cbb344375f01 (table rebalancer_dynamic_table [id=75970b92fa1644ec9b8346a7970a2e7d]). Expected version 1 got 0
I20240502 07:59:06.437480 28985 tablet.cc:1722] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:06.437866 28985 tablet.cc:1620] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:06.440194 28984 tablet.cc:1722] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:06.440558 28984 tablet.cc:1620] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:06.445241 22228 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=75970b92fa1644ec9b8346a7970a2e7d] alter complete (version 1)
I20240502 07:59:06.446771 22225 catalog_manager.cc:5576] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995) does not have the latest schema for tablet 5f78e3e5cb144f8783c7cbb344375f01 (table rebalancer_dynamic_table [id=75970b92fa1644ec9b8346a7970a2e7d]). Expected version 1 got 0
I20240502 07:59:06.447299 28986 tablet.cc:1722] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:06.447788 28986 tablet.cc:1620] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:06.463073 22225 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_26"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:06.470240 24800 tablet_service.cc:1467] Processing CreateTablet for tablet e75b8171bf164f748fce5a71d86c41bf (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=e0a920fd5961477b92891b52ec936746]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:06.470588 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e75b8171bf164f748fce5a71d86c41bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.470765 25062 tablet_service.cc:1467] Processing CreateTablet for tablet 05911d56b0344eb8a53aba5f4bf0170c (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=e0a920fd5961477b92891b52ec936746]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:06.470999 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05911d56b0344eb8a53aba5f4bf0170c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.472630 28823 tablet_bootstrap.cc:492] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:06.473021 25063 tablet_service.cc:1467] Processing CreateTablet for tablet e75b8171bf164f748fce5a71d86c41bf (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=e0a920fd5961477b92891b52ec936746]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:06.473235 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e75b8171bf164f748fce5a71d86c41bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.474014 28823 tablet_bootstrap.cc:654] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:06.474619 28822 tablet_bootstrap.cc:492] T e75b8171bf164f748fce5a71d86c41bf P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:06.475489 28822 tablet_bootstrap.cc:654] T e75b8171bf164f748fce5a71d86c41bf P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:06.476445 25062 tablet_service.cc:1467] Processing CreateTablet for tablet a7da69fb73f94ce0b8c24599c8ae4d15 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=e0a920fd5961477b92891b52ec936746]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:06.476693 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7da69fb73f94ce0b8c24599c8ae4d15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.477046 25195 tablet_service.cc:1467] Processing CreateTablet for tablet e75b8171bf164f748fce5a71d86c41bf (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=e0a920fd5961477b92891b52ec936746]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:06.477295 28823 tablet_bootstrap.cc:492] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:06.477311 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e75b8171bf164f748fce5a71d86c41bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.477272 25194 tablet_service.cc:1467] Processing CreateTablet for tablet 05911d56b0344eb8a53aba5f4bf0170c (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=e0a920fd5961477b92891b52ec936746]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:06.477406 28823 ts_tablet_manager.cc:1367] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 07:59:06.477532 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05911d56b0344eb8a53aba5f4bf0170c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.477934 28823 raft_consensus.cc:348] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:06.478042 28823 raft_consensus.cc:374] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:06.478085 28823 raft_consensus.cc:729] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:06.478210 28823 consensus_queue.cc:260] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:06.478473 28823 ts_tablet_manager.cc:1398] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:06.478730 25193 tablet_service.cc:1467] Processing CreateTablet for tablet a7da69fb73f94ce0b8c24599c8ae4d15 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=e0a920fd5961477b92891b52ec936746]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:06.478950 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7da69fb73f94ce0b8c24599c8ae4d15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.478878 24931 tablet_service.cc:1467] Processing CreateTablet for tablet a7da69fb73f94ce0b8c24599c8ae4d15 (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=e0a920fd5961477b92891b52ec936746]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:06.479048 28823 tablet_bootstrap.cc:492] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:06.479139 24931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7da69fb73f94ce0b8c24599c8ae4d15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.480000 28823 tablet_bootstrap.cc:654] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:06.482144 28652 tablet_bootstrap.cc:492] T a7da69fb73f94ce0b8c24599c8ae4d15 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:06.483197 28652 tablet_bootstrap.cc:654] T a7da69fb73f94ce0b8c24599c8ae4d15 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:06.484587 28822 tablet_bootstrap.cc:492] T e75b8171bf164f748fce5a71d86c41bf P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:06.484685 28822 ts_tablet_manager.cc:1367] T e75b8171bf164f748fce5a71d86c41bf P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 07:59:06.485239 28822 raft_consensus.cc:348] T e75b8171bf164f748fce5a71d86c41bf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:06.485419 28822 raft_consensus.cc:374] T e75b8171bf164f748fce5a71d86c41bf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:06.485463 28822 raft_consensus.cc:729] T e75b8171bf164f748fce5a71d86c41bf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:06.485617 28822 consensus_queue.cc:260] T e75b8171bf164f748fce5a71d86c41bf P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:06.486711 28822 ts_tablet_manager.cc:1398] T e75b8171bf164f748fce5a71d86c41bf P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:06.486744 28652 tablet_bootstrap.cc:492] T a7da69fb73f94ce0b8c24599c8ae4d15 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:06.486840 28652 ts_tablet_manager.cc:1367] T a7da69fb73f94ce0b8c24599c8ae4d15 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20240502 07:59:06.487241 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 05911d56b0344eb8a53aba5f4bf0170c (DEFAULT_TABLE table=rebalancer_extra_table_26 [id=e0a920fd5961477b92891b52ec936746]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:06.487514 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05911d56b0344eb8a53aba5f4bf0170c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.487896 28652 raft_consensus.cc:348] T a7da69fb73f94ce0b8c24599c8ae4d15 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:06.488004 28652 raft_consensus.cc:374] T a7da69fb73f94ce0b8c24599c8ae4d15 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:06.488039 28652 raft_consensus.cc:729] T a7da69fb73f94ce0b8c24599c8ae4d15 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:06.488070 28823 tablet_bootstrap.cc:492] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:06.488149 28823 ts_tablet_manager.cc:1367] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240502 07:59:06.488155 28652 consensus_queue.cc:260] T a7da69fb73f94ce0b8c24599c8ae4d15 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:06.488446 28652 ts_tablet_manager.cc:1398] T a7da69fb73f94ce0b8c24599c8ae4d15 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:06.488880 28464 raft_consensus.cc:482] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:06.488966 28464 raft_consensus.cc:504] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:06.489857 28822 tablet_bootstrap.cc:492] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:06.489849 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05911d56b0344eb8a53aba5f4bf0170c" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:06.489315 28464 leader_election.cc:288] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
W20240502 07:59:06.489917 25019 leader_election.cc:341] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Not found: Tablet not found: 05911d56b0344eb8a53aba5f4bf0170c
W20240502 07:59:06.491187 25020 leader_election.cc:341] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): Illegal state: must be running to vote when last-logged opid is not known
I20240502 07:59:06.491235 25020 leader_election.cc:302] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055; no voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:06.491609 28464 raft_consensus.cc:2726] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 07:59:06.491669 28822 tablet_bootstrap.cc:654] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:06.492337 28823 raft_consensus.cc:348] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:06.493276 28823 raft_consensus.cc:374] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:06.493393 28823 raft_consensus.cc:729] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:06.493424 28822 tablet_bootstrap.cc:492] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:06.493559 28822 ts_tablet_manager.cc:1367] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:06.493636 28823 consensus_queue.cc:260] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:06.494098 28823 ts_tablet_manager.cc:1398] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 07:59:06.494338 28823 tablet_bootstrap.cc:492] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:06.494815 28822 raft_consensus.cc:348] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:06.494958 28822 raft_consensus.cc:374] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:06.495028 28822 raft_consensus.cc:729] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:06.495725 28823 tablet_bootstrap.cc:654] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:06.495177 28822 consensus_queue.cc:260] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:06.496655 28822 ts_tablet_manager.cc:1398] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:06.498021 28995 tablet_bootstrap.cc:492] T e75b8171bf164f748fce5a71d86c41bf P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:06.498088 28823 tablet_bootstrap.cc:492] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:06.498167 28823 ts_tablet_manager.cc:1367] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:06.498605 28823 raft_consensus.cc:348] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:06.498697 28823 raft_consensus.cc:374] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:06.498726 28823 raft_consensus.cc:729] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:06.498832 28823 consensus_queue.cc:260] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:06.499074 28823 ts_tablet_manager.cc:1398] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:06.500985 28995 tablet_bootstrap.cc:654] T e75b8171bf164f748fce5a71d86c41bf P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:06.502338 28995 tablet_bootstrap.cc:492] T e75b8171bf164f748fce5a71d86c41bf P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:06.502542 28995 ts_tablet_manager.cc:1367] T e75b8171bf164f748fce5a71d86c41bf P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:06.503010 28995 raft_consensus.cc:348] T e75b8171bf164f748fce5a71d86c41bf P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:06.503190 28995 raft_consensus.cc:374] T e75b8171bf164f748fce5a71d86c41bf P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:06.503269 28995 raft_consensus.cc:729] T e75b8171bf164f748fce5a71d86c41bf P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:06.503432 28995 consensus_queue.cc:260] T e75b8171bf164f748fce5a71d86c41bf P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:06.503720 28995 ts_tablet_manager.cc:1398] T e75b8171bf164f748fce5a71d86c41bf P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:06.504009 28995 tablet_bootstrap.cc:492] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:06.505234 28995 tablet_bootstrap.cc:654] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:06.506294 28995 tablet_bootstrap.cc:492] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:06.506386 28995 ts_tablet_manager.cc:1367] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:06.508594 28995 raft_consensus.cc:348] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:06.508695 28995 raft_consensus.cc:374] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:06.508730 28995 raft_consensus.cc:729] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:06.508872 28995 consensus_queue.cc:260] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:06.509177 28995 ts_tablet_manager.cc:1398] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:06.509380 28995 tablet_bootstrap.cc:492] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:06.510227 28995 tablet_bootstrap.cc:654] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:06.511914 28995 tablet_bootstrap.cc:492] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:06.512012 28995 ts_tablet_manager.cc:1367] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:06.512418 28995 raft_consensus.cc:348] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:06.512534 28995 raft_consensus.cc:374] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:06.512583 28995 raft_consensus.cc:729] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:06.512707 28995 consensus_queue.cc:260] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:06.513211 28995 ts_tablet_manager.cc:1398] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:06.521239 28851 raft_consensus.cc:482] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:06.521409 28851 raft_consensus.cc:504] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:06.521839 28851 leader_election.cc:288] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:06.522315 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05911d56b0344eb8a53aba5f4bf0170c" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:06.522346 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05911d56b0344eb8a53aba5f4bf0170c" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:06.522445 25215 raft_consensus.cc:2454] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:06.522472 25084 raft_consensus.cc:2454] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:06.522747 24757 leader_election.cc:302] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:06.522965 28851 raft_consensus.cc:2781] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:06.523031 28851 raft_consensus.cc:482] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:06.523084 28851 raft_consensus.cc:3037] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:06.524262 28851 raft_consensus.cc:504] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:06.524643 28851 leader_election.cc:288] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:06.525382 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05911d56b0344eb8a53aba5f4bf0170c" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:06.525451 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05911d56b0344eb8a53aba5f4bf0170c" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:06.525514 25215 raft_consensus.cc:3037] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:06.525746 25084 raft_consensus.cc:3037] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:06.526650 25215 raft_consensus.cc:2454] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:06.526667 25084 raft_consensus.cc:2454] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:06.527194 24757 leader_election.cc:302] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:06.527560 28851 raft_consensus.cc:2781] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:06.527658 28851 raft_consensus.cc:686] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:59:06.527802 28851 consensus_queue.cc:237] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:06.531581 22227 catalog_manager.cc:5484] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:06.551091 22227 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:06.557989 28984 tablet.cc:1722] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:06.558326 28984 tablet.cc:1620] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:06.558399 28986 tablet.cc:1722] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:06.558702 28986 tablet.cc:1620] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:06.559474 28985 tablet.cc:1722] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:06.559783 28985 tablet.cc:1620] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:06.566964 22221 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=75970b92fa1644ec9b8346a7970a2e7d] alter complete (version 2)
I20240502 07:59:06.569496 28972 raft_consensus.cc:482] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:06.569613 28972 raft_consensus.cc:504] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:06.570022 28972 leader_election.cc:288] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:06.570317 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7da69fb73f94ce0b8c24599c8ae4d15" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:06.570477 24951 raft_consensus.cc:2454] T a7da69fb73f94ce0b8c24599c8ae4d15 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:06.570787 25151 leader_election.cc:302] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:06.570799 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7da69fb73f94ce0b8c24599c8ae4d15" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:06.570933 25084 raft_consensus.cc:2454] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:06.571636 28972 raft_consensus.cc:2781] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:06.571719 28972 raft_consensus.cc:482] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:06.571758 28972 raft_consensus.cc:3037] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:06.572830 28972 raft_consensus.cc:504] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:06.573192 28972 leader_election.cc:288] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:06.575466 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7da69fb73f94ce0b8c24599c8ae4d15" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:06.575592 24951 raft_consensus.cc:3037] T a7da69fb73f94ce0b8c24599c8ae4d15 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:06.576665 24951 raft_consensus.cc:2454] T a7da69fb73f94ce0b8c24599c8ae4d15 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:06.576961 25151 leader_election.cc:302] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:06.577791 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7da69fb73f94ce0b8c24599c8ae4d15" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:06.577913 25084 raft_consensus.cc:3037] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:06.578815 25084 raft_consensus.cc:2454] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:06.580013 28972 raft_consensus.cc:2781] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:06.580124 28972 raft_consensus.cc:686] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:06.580243 28972 consensus_queue.cc:237] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:06.582257 22225 catalog_manager.cc:5484] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:06.643914 29034 raft_consensus.cc:482] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:06.644040 29034 raft_consensus.cc:504] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:06.644465 29034 leader_election.cc:288] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:06.644819 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e75b8171bf164f748fce5a71d86c41bf" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:06.644858 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e75b8171bf164f748fce5a71d86c41bf" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:06.644968 24820 raft_consensus.cc:2454] T e75b8171bf164f748fce5a71d86c41bf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:06.644979 25215 raft_consensus.cc:2454] T e75b8171bf164f748fce5a71d86c41bf P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:06.645232 25020 leader_election.cc:302] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:06.645421 29034 raft_consensus.cc:2781] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:06.645484 29034 raft_consensus.cc:482] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:06.645539 29034 raft_consensus.cc:3037] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:06.646703 29034 raft_consensus.cc:504] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:06.646979 29034 leader_election.cc:288] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:06.647325 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e75b8171bf164f748fce5a71d86c41bf" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:06.647441 25215 raft_consensus.cc:3037] T e75b8171bf164f748fce5a71d86c41bf P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:06.647462 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e75b8171bf164f748fce5a71d86c41bf" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:06.647581 24820 raft_consensus.cc:3037] T e75b8171bf164f748fce5a71d86c41bf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:06.648334 25215 raft_consensus.cc:2454] T e75b8171bf164f748fce5a71d86c41bf P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:06.648602 24820 raft_consensus.cc:2454] T e75b8171bf164f748fce5a71d86c41bf P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:06.648712 25020 leader_election.cc:302] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:06.648866 29034 raft_consensus.cc:2781] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:06.648949 29034 raft_consensus.cc:686] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:06.649082 29034 consensus_queue.cc:237] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:06.650962 22225 catalog_manager.cc:5484] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:06.670681 22225 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:06.678323 24931 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:06.679186 28984 tablet.cc:1722] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:06.680091 28986 tablet.cc:1722] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:06.680748 25063 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:06.684927 24800 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:06.685266 25194 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:06.686480 28985 tablet.cc:1722] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:06.701391 22225 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=75970b92fa1644ec9b8346a7970a2e7d] alter complete (version 3)
I20240502 07:59:06.704600 22351 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:06.717851 22614 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:06.718788 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:06.721648 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:06.789475 24821 consensus_queue.cc:237] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:06.790875 22372 raft_consensus.cc:1261] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer 4b4e7eeb7f3f42119c0363a79febc54b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:06.790956 22764 raft_consensus.cc:1261] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer 4b4e7eeb7f3f42119c0363a79febc54b: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:06.791302 28851 consensus_queue.cc:1035] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:06.791519 28570 consensus_queue.cc:1035] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:06.793264 28570 raft_consensus.cc:2932] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:06.793720 22764 raft_consensus.cc:2932] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:06.794171 22372 raft_consensus.cc:2932] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:06.796088 22228 catalog_manager.cc:5484] T 01d9a73be1fd472db39860e538b9c97e P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:06.798759 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet 01d9a73be1fd472db39860e538b9c97e with delete_type TABLET_DATA_TOMBSTONED (TS 227eec4b5f014d349661f6e0b39b9055 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:06.799197 29067 tablet_replica.cc:330] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:06.799396 29067 raft_consensus.cc:2229] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:06.799589 29067 raft_consensus.cc:2258] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:06.800206 29067 ts_tablet_manager.cc:1875] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:06.802429 29067 ts_tablet_manager.cc:1888] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:06.802515 29067 log.cc:1196] T 01d9a73be1fd472db39860e538b9c97e P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/01d9a73be1fd472db39860e538b9c97e
I20240502 07:59:06.803061 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet 01d9a73be1fd472db39860e538b9c97e (table rebalancer_extra_table_4 [id=7fe3829782fb4a98912517ae49651ba5]) successfully deleted
I20240502 07:59:06.807330 22221 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:06.807477 22221 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:06.809543 22221 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 5f78e3e5cb144f8783c7cbb344375f01
I20240502 07:59:06.810238 24800 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f78e3e5cb144f8783c7cbb344375f01 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:06 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:59:06.810312 24931 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f78e3e5cb144f8783c7cbb344375f01 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:06 UTC) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:06.810634 29080 tablet_replica.cc:330] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:06.810710 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f78e3e5cb144f8783c7cbb344375f01 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:06 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:06.810815 29080 raft_consensus.cc:2229] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:06.810878 29067 tablet_replica.cc:330] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:06.810945 29080 raft_consensus.cc:2258] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:06.810959 29079 tablet_replica.cc:330] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:59:06.811007 29067 raft_consensus.cc:2229] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:06.811096 29079 raft_consensus.cc:2229] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:06.811193 29067 raft_consensus.cc:2258] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:06.811220 29079 raft_consensus.cc:2258] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:06.811589 29080 ts_tablet_manager.cc:1875] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:06.811895 29067 ts_tablet_manager.cc:1875] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:06.811980 29079 ts_tablet_manager.cc:1875] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:06.813725 29067 ts_tablet_manager.cc:1888] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:06.813802 29067 log.cc:1196] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/5f78e3e5cb144f8783c7cbb344375f01
I20240502 07:59:06.814065 29067 ts_tablet_manager.cc:1909] T 5f78e3e5cb144f8783c7cbb344375f01 P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:59:06.814150 29079 ts_tablet_manager.cc:1888] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:06.814211 29079 log.cc:1196] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/5f78e3e5cb144f8783c7cbb344375f01
I20240502 07:59:06.814574 29079 ts_tablet_manager.cc:1909] T 5f78e3e5cb144f8783c7cbb344375f01 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:59:06.815495 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet 5f78e3e5cb144f8783c7cbb344375f01 (table rebalancer_dynamic_table_new_name [id=75970b92fa1644ec9b8346a7970a2e7d]) successfully deleted
I20240502 07:59:06.815713 29080 ts_tablet_manager.cc:1888] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:06.815858 29080 log.cc:1196] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/5f78e3e5cb144f8783c7cbb344375f01
I20240502 07:59:06.815989 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet 5f78e3e5cb144f8783c7cbb344375f01 (table rebalancer_dynamic_table_new_name [id=75970b92fa1644ec9b8346a7970a2e7d]) successfully deleted
I20240502 07:59:06.816205 29080 ts_tablet_manager.cc:1909] T 5f78e3e5cb144f8783c7cbb344375f01 P 1aecef81733c47b3b468504934357231: Deleting consensus metadata
I20240502 07:59:06.816917 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet 5f78e3e5cb144f8783c7cbb344375f01 (table rebalancer_dynamic_table_new_name [id=75970b92fa1644ec9b8346a7970a2e7d]) successfully deleted
I20240502 07:59:06.856762 22764 raft_consensus.cc:1261] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:06.857352 28815 consensus_queue.cc:1035] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:06.866518 22499 raft_consensus.cc:1261] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c [term 2 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:06.868217 28686 consensus_queue.cc:1035] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:06.881340 22634 raft_consensus.cc:1261] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 2 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:06.882078 28815 consensus_queue.cc:1035] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:06.908077 25063 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:06.910358 24931 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:06.911623 22223 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:06.914453 24800 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:06.916188 25194 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:06.924233 22743 tablet_service.cc:1467] Processing CreateTablet for tablet e84280470d9c4047bbc471aac904dcd2 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a85bcb9494e044348f00e0aff1f0cbb1]), partition=
I20240502 07:59:06.924384 22351 tablet_service.cc:1467] Processing CreateTablet for tablet e84280470d9c4047bbc471aac904dcd2 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a85bcb9494e044348f00e0aff1f0cbb1]), partition=
I20240502 07:59:06.924506 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e84280470d9c4047bbc471aac904dcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.924649 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e84280470d9c4047bbc471aac904dcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.926904 22483 tablet_service.cc:1467] Processing CreateTablet for tablet e84280470d9c4047bbc471aac904dcd2 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a85bcb9494e044348f00e0aff1f0cbb1]), partition=
I20240502 07:59:06.927138 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e84280470d9c4047bbc471aac904dcd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:06.929555 29130 tablet_bootstrap.cc:492] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:06.930502 29130 tablet_bootstrap.cc:654] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:06.931603 29130 tablet_bootstrap.cc:492] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:06.931684 29130 ts_tablet_manager.cc:1367] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:06.932104 29130 raft_consensus.cc:348] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:06.932215 29130 raft_consensus.cc:374] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:06.932246 29130 raft_consensus.cc:729] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:06.932358 29130 consensus_queue.cc:260] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:06.932653 29130 ts_tablet_manager.cc:1398] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:06.937665 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:06.946453 29129 tablet_bootstrap.cc:492] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:06.947618 29129 tablet_bootstrap.cc:654] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:06.949474 29128 tablet_bootstrap.cc:492] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:06.950613 29128 tablet_bootstrap.cc:654] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:06.951406 22351 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:06.951967 22614 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:06.974573 29129 tablet_bootstrap.cc:492] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:06.975353 29129 ts_tablet_manager.cc:1367] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20240502 07:59:06.975925 29128 tablet_bootstrap.cc:492] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:06.976034 29128 ts_tablet_manager.cc:1367] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20240502 07:59:06.976068 29129 raft_consensus.cc:348] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:06.976198 29129 raft_consensus.cc:374] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:06.976274 29129 raft_consensus.cc:729] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:06.976442 29129 consensus_queue.cc:260] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:06.976774 29129 ts_tablet_manager.cc:1398] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:06.977265 29128 raft_consensus.cc:348] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:06.977424 29128 raft_consensus.cc:374] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:06.977489 29128 raft_consensus.cc:729] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:06.977617 29128 consensus_queue.cc:260] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:06.977983 29128 ts_tablet_manager.cc:1398] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:06.980105 22743 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:07.006392 28593 consensus_queue.cc:1035] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:07.006713 28972 consensus_queue.cc:1035] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:07.021289 28773 raft_consensus.cc:482] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:07.021447 28773 raft_consensus.cc:504] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:07.021883 28773 leader_election.cc:288] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:07.022353 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e84280470d9c4047bbc471aac904dcd2" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:07.022373 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e84280470d9c4047bbc471aac904dcd2" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:07.022503 22501 raft_consensus.cc:2454] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:07.022507 22372 raft_consensus.cc:2454] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:07.022817 22701 leader_election.cc:302] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:07.022994 28773 raft_consensus.cc:2781] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:07.023051 28773 raft_consensus.cc:482] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:07.023097 28773 raft_consensus.cc:3037] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:07.024022 28773 raft_consensus.cc:504] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:07.024358 28773 leader_election.cc:288] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:07.024416 22501 consensus_queue.cc:237] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:07.024634 22499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e84280470d9c4047bbc471aac904dcd2" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:07.024765 22499 raft_consensus.cc:3037] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:07.024976 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e84280470d9c4047bbc471aac904dcd2" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:07.025065 22372 raft_consensus.cc:3037] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:07.025892 22499 raft_consensus.cc:2454] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:07.026021 22372 raft_consensus.cc:2454] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:07.026262 22701 leader_election.cc:302] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:07.026425 29153 raft_consensus.cc:2781] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:07.026494 29153 raft_consensus.cc:686] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:59:07.026633 29153 consensus_queue.cc:237] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:07.026851 22632 raft_consensus.cc:1261] T 847f3a2601d44898a14a0b35409981ad P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:07.027066 25215 raft_consensus.cc:1261] T 847f3a2601d44898a14a0b35409981ad P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:07.027297 28757 consensus_queue.cc:1035] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:07.027633 28757 consensus_queue.cc:1035] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:07.029392 28757 raft_consensus.cc:2932] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:07.030908 22228 catalog_manager.cc:5484] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:07.031057 22632 raft_consensus.cc:2932] T 847f3a2601d44898a14a0b35409981ad P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:07.033124 22223 catalog_manager.cc:5484] T 847f3a2601d44898a14a0b35409981ad P fabddb16c85944648a710ec52fc7244c reported cstate change: config changed from index 4 to 5, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:07.036944 22743 tablet_service.cc:1514] Processing DeleteTablet for tablet 847f3a2601d44898a14a0b35409981ad with delete_type TABLET_DATA_TOMBSTONED (TS f00f3a1876444ad2bf9c919068ab8014 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:07.037483 29162 tablet_replica.cc:330] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:07.037638 29162 raft_consensus.cc:2229] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:07.037811 29162 raft_consensus.cc:2258] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:07.037873 28570 consensus_queue.cc:1035] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:07.038353 29162 ts_tablet_manager.cc:1875] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:07.040558 29162 ts_tablet_manager.cc:1888] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:07.040632 29162 log.cc:1196] T 847f3a2601d44898a14a0b35409981ad P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/847f3a2601d44898a14a0b35409981ad
I20240502 07:59:07.041152 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 847f3a2601d44898a14a0b35409981ad (table rebalance_test_table_19 [id=4e87626760fa4d8098e55dfa6d1c55a9]) successfully deleted
I20240502 07:59:07.041183 28851 consensus_queue.cc:1035] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:07.047238 25215 raft_consensus.cc:2932] T 847f3a2601d44898a14a0b35409981ad P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:07.079545 29034 consensus_queue.cc:1035] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:07.087285 28464 consensus_queue.cc:1035] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:07.147835 25194 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:07.150816 25063 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:07.157965 22614 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:07.159555 22743 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:07.159691 22227 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:07.161700 22351 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:07.162120 24800 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:07.165230 22372 raft_consensus.cc:1261] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:07.165297 22499 raft_consensus.cc:1261] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:07.165719 29153 consensus_queue.cc:1035] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:07.167129 29153 consensus_queue.cc:1035] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:07.169971 22226 catalog_manager.cc:5576] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535) does not have the latest schema for tablet e84280470d9c4047bbc471aac904dcd2 (table rebalancer_dynamic_table [id=a85bcb9494e044348f00e0aff1f0cbb1]). Expected version 1 got 0
I20240502 07:59:07.170555 22227 catalog_manager.cc:5576] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625) does not have the latest schema for tablet e84280470d9c4047bbc471aac904dcd2 (table rebalancer_dynamic_table [id=a85bcb9494e044348f00e0aff1f0cbb1]). Expected version 1 got 0
I20240502 07:59:07.171118 29219 tablet.cc:1722] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:07.171480 29219 tablet.cc:1620] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c: MemRowSet was empty: no flush needed.
I20240502 07:59:07.171814 24931 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:07.183909 22227 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=a85bcb9494e044348f00e0aff1f0cbb1] alter complete (version 1)
I20240502 07:59:07.185132 22226 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_28"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:07.191885 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 8752733ffcbd4667b586366c09d85e4f (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=4f0b5c7600f3499cb8a39cd5d49bbf72]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:07.192042 25194 tablet_service.cc:1467] Processing CreateTablet for tablet 8752733ffcbd4667b586366c09d85e4f (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=4f0b5c7600f3499cb8a39cd5d49bbf72]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:07.192179 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8752733ffcbd4667b586366c09d85e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:07.192317 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8752733ffcbd4667b586366c09d85e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:07.193646 25062 tablet_service.cc:1467] Processing CreateTablet for tablet 8752733ffcbd4667b586366c09d85e4f (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=4f0b5c7600f3499cb8a39cd5d49bbf72]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:07.193934 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8752733ffcbd4667b586366c09d85e4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:07.194350 25193 tablet_service.cc:1467] Processing CreateTablet for tablet 4cd6c14c7c8b4ad29f2107e94fd11acc (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=4f0b5c7600f3499cb8a39cd5d49bbf72]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:07.194563 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cd6c14c7c8b4ad29f2107e94fd11acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:07.196180 29222 tablet_bootstrap.cc:492] T 8752733ffcbd4667b586366c09d85e4f P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:07.196570 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 4cd6c14c7c8b4ad29f2107e94fd11acc (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=4f0b5c7600f3499cb8a39cd5d49bbf72]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:07.196827 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cd6c14c7c8b4ad29f2107e94fd11acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:07.196887 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 351f047b9b53400b8cc6286359edb813 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=4f0b5c7600f3499cb8a39cd5d49bbf72]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:07.197119 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 351f047b9b53400b8cc6286359edb813. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:07.197432 29222 tablet_bootstrap.cc:654] T 8752733ffcbd4667b586366c09d85e4f P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:07.198851 24931 tablet_service.cc:1467] Processing CreateTablet for tablet 351f047b9b53400b8cc6286359edb813 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=4f0b5c7600f3499cb8a39cd5d49bbf72]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:07.199092 29224 tablet_bootstrap.cc:492] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:07.199105 24931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 351f047b9b53400b8cc6286359edb813. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:07.200018 29224 tablet_bootstrap.cc:654] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:07.200718 25062 tablet_service.cc:1467] Processing CreateTablet for tablet 4cd6c14c7c8b4ad29f2107e94fd11acc (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=4f0b5c7600f3499cb8a39cd5d49bbf72]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:07.200968 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cd6c14c7c8b4ad29f2107e94fd11acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:07.201887 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 351f047b9b53400b8cc6286359edb813 (DEFAULT_TABLE table=rebalancer_extra_table_28 [id=4f0b5c7600f3499cb8a39cd5d49bbf72]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:07.202147 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 351f047b9b53400b8cc6286359edb813. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:07.204476 29221 tablet_bootstrap.cc:492] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:07.205476 29221 tablet_bootstrap.cc:654] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:07.205894 29129 tablet_bootstrap.cc:492] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:07.206872 29129 tablet_bootstrap.cc:654] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:07.207751 29222 tablet_bootstrap.cc:492] T 8752733ffcbd4667b586366c09d85e4f P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:07.207842 29222 ts_tablet_manager.cc:1367] T 8752733ffcbd4667b586366c09d85e4f P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20240502 07:59:07.208101 29129 tablet_bootstrap.cc:492] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:07.208194 29129 ts_tablet_manager.cc:1367] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:07.208351 29222 raft_consensus.cc:348] T 8752733ffcbd4667b586366c09d85e4f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:07.208529 29222 raft_consensus.cc:374] T 8752733ffcbd4667b586366c09d85e4f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:07.208613 29222 raft_consensus.cc:729] T 8752733ffcbd4667b586366c09d85e4f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:07.208652 29129 raft_consensus.cc:348] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:07.208761 29129 raft_consensus.cc:374] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:07.208799 29129 raft_consensus.cc:729] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:07.208927 29129 consensus_queue.cc:260] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:07.208920 29222 consensus_queue.cc:260] T 8752733ffcbd4667b586366c09d85e4f P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:07.209211 29129 ts_tablet_manager.cc:1398] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:07.209297 29222 ts_tablet_manager.cc:1398] T 8752733ffcbd4667b586366c09d85e4f P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:07.209543 29222 tablet_bootstrap.cc:492] T 351f047b9b53400b8cc6286359edb813 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:07.211131 22227 catalog_manager.cc:5576] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535) does not have the latest schema for tablet e84280470d9c4047bbc471aac904dcd2 (table rebalancer_dynamic_table [id=a85bcb9494e044348f00e0aff1f0cbb1]). Expected version 1 got 0
I20240502 07:59:07.214504 29222 tablet_bootstrap.cc:654] T 351f047b9b53400b8cc6286359edb813 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:07.216913 29221 tablet_bootstrap.cc:492] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:07.217162 29221 ts_tablet_manager.cc:1367] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 07:59:07.217824 29221 raft_consensus.cc:348] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:07.217994 29221 raft_consensus.cc:374] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:07.218060 29221 raft_consensus.cc:729] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:07.218214 29221 consensus_queue.cc:260] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:07.219226 29221 ts_tablet_manager.cc:1398] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:07.219448 29221 tablet_bootstrap.cc:492] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:07.220140 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:07.220464 29221 tablet_bootstrap.cc:654] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:07.221568 29221 tablet_bootstrap.cc:492] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:07.221652 29221 ts_tablet_manager.cc:1367] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:07.222071 29221 raft_consensus.cc:348] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.222174 29221 raft_consensus.cc:374] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:07.222203 29221 raft_consensus.cc:729] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:07.222318 29221 consensus_queue.cc:260] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.222584 29221 ts_tablet_manager.cc:1398] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:07.228406 29220 tablet_bootstrap.cc:492] T 8752733ffcbd4667b586366c09d85e4f P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:07.229455 29220 tablet_bootstrap.cc:654] T 8752733ffcbd4667b586366c09d85e4f P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:07.230125 29218 tablet.cc:1722] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:07.230465 29218 tablet.cc:1620] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:59:07.231724 29222 tablet_bootstrap.cc:492] T 351f047b9b53400b8cc6286359edb813 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:07.231814 29222 ts_tablet_manager.cc:1367] T 351f047b9b53400b8cc6286359edb813 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.002s
I20240502 07:59:07.232277 29222 raft_consensus.cc:348] T 351f047b9b53400b8cc6286359edb813 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:07.232391 29222 raft_consensus.cc:374] T 351f047b9b53400b8cc6286359edb813 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:07.232424 29222 raft_consensus.cc:729] T 351f047b9b53400b8cc6286359edb813 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:07.232563 29222 consensus_queue.cc:260] T 351f047b9b53400b8cc6286359edb813 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:07.232847 29222 ts_tablet_manager.cc:1398] T 351f047b9b53400b8cc6286359edb813 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:07.233103 29222 tablet_bootstrap.cc:492] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:07.233508 29231 tablet.cc:1722] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:07.233871 29231 tablet.cc:1620] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:07.234054 29222 tablet_bootstrap.cc:654] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:07.236402 29222 tablet_bootstrap.cc:492] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:07.236491 29222 ts_tablet_manager.cc:1367] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:07.237130 29222 raft_consensus.cc:348] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.237239 29222 raft_consensus.cc:374] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:07.237277 29222 raft_consensus.cc:729] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:07.237411 29222 consensus_queue.cc:260] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.237653 29222 ts_tablet_manager.cc:1398] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:07.241935 29224 tablet_bootstrap.cc:492] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:07.242084 29224 ts_tablet_manager.cc:1367] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.043s user 0.002s sys 0.000s
I20240502 07:59:07.242585 29224 raft_consensus.cc:348] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.242756 29224 raft_consensus.cc:374] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:07.242825 29224 raft_consensus.cc:729] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:07.242977 29224 consensus_queue.cc:260] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.243309 29224 ts_tablet_manager.cc:1398] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:07.265209 29220 tablet_bootstrap.cc:492] T 8752733ffcbd4667b586366c09d85e4f P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:07.265370 29220 ts_tablet_manager.cc:1367] T 8752733ffcbd4667b586366c09d85e4f P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.002s
I20240502 07:59:07.265862 29220 raft_consensus.cc:348] T 8752733ffcbd4667b586366c09d85e4f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:07.265986 29220 raft_consensus.cc:374] T 8752733ffcbd4667b586366c09d85e4f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:07.266023 29220 raft_consensus.cc:729] T 8752733ffcbd4667b586366c09d85e4f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:07.266145 29220 consensus_queue.cc:260] T 8752733ffcbd4667b586366c09d85e4f P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:07.266479 29220 ts_tablet_manager.cc:1398] T 8752733ffcbd4667b586366c09d85e4f P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:07.266669 29220 tablet_bootstrap.cc:492] T 351f047b9b53400b8cc6286359edb813 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:07.267668 29220 tablet_bootstrap.cc:654] T 351f047b9b53400b8cc6286359edb813 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:07.270823 29220 tablet_bootstrap.cc:492] T 351f047b9b53400b8cc6286359edb813 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:07.270956 29220 ts_tablet_manager.cc:1367] T 351f047b9b53400b8cc6286359edb813 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 07:59:07.271503 29220 raft_consensus.cc:348] T 351f047b9b53400b8cc6286359edb813 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:07.271771 29220 raft_consensus.cc:374] T 351f047b9b53400b8cc6286359edb813 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:07.271831 29220 raft_consensus.cc:729] T 351f047b9b53400b8cc6286359edb813 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:07.271968 29220 consensus_queue.cc:260] T 351f047b9b53400b8cc6286359edb813 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:07.272257 29220 ts_tablet_manager.cc:1398] T 351f047b9b53400b8cc6286359edb813 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:07.287722 22226 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:07.293735 24951 consensus_queue.cc:237] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.293874 29218 tablet.cc:1722] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:07.294170 29218 tablet.cc:1620] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:59:07.294461 29231 tablet.cc:1722] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:07.294778 29231 tablet.cc:1620] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:07.294956 24821 raft_consensus.cc:1261] T 4840fa82fcb241f391c4890fb735fe9a P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:07.295315 29219 tablet.cc:1722] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:07.295603 29219 tablet.cc:1620] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c: MemRowSet was empty: no flush needed.
I20240502 07:59:07.296070 28815 consensus_queue.cc:1035] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:07.297266 25083 raft_consensus.cc:1261] T 4840fa82fcb241f391c4890fb735fe9a P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:07.299548 28815 consensus_queue.cc:1035] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:07.301803 29113 raft_consensus.cc:2932] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:07.301981 22226 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=a85bcb9494e044348f00e0aff1f0cbb1] alter complete (version 2)
I20240502 07:59:07.302342 25083 raft_consensus.cc:2932] T 4840fa82fcb241f391c4890fb735fe9a P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:07.304322 22227 catalog_manager.cc:5484] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 reported cstate change: config changed from index 2 to 3, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) evicted. New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:07.304780 24821 raft_consensus.cc:2932] T 4840fa82fcb241f391c4890fb735fe9a P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:07.307489 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet 4840fa82fcb241f391c4890fb735fe9a with delete_type TABLET_DATA_TOMBSTONED (TS b533a7d8d97943609c6e28e3e36b78a4 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:07.308706 29253 tablet_replica.cc:330] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:07.308920 29253 raft_consensus.cc:2229] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:07.309057 29253 raft_consensus.cc:2258] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:07.309662 29253 ts_tablet_manager.cc:1875] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:07.312011 29253 ts_tablet_manager.cc:1888] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:07.312088 29253 log.cc:1196] T 4840fa82fcb241f391c4890fb735fe9a P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/4840fa82fcb241f391c4890fb735fe9a
I20240502 07:59:07.312707 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 4840fa82fcb241f391c4890fb735fe9a (table rebalancer_extra_table_2 [id=5b6a061be2624cb1852eb43e3a4cb66b]) successfully deleted
I20240502 07:59:07.387837 28593 raft_consensus.cc:482] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:07.387961 28593 raft_consensus.cc:504] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:07.388375 28593 leader_election.cc:288] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:07.388971 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8752733ffcbd4667b586366c09d85e4f" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:07.389122 24951 raft_consensus.cc:2454] T 8752733ffcbd4667b586366c09d85e4f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:07.389504 25151 leader_election.cc:302] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:07.389667 28593 raft_consensus.cc:2781] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:07.389719 28593 raft_consensus.cc:482] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:07.389699 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8752733ffcbd4667b586366c09d85e4f" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:07.389763 28593 raft_consensus.cc:3037] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:07.389838 25083 raft_consensus.cc:2454] T 8752733ffcbd4667b586366c09d85e4f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:07.390769 28593 raft_consensus.cc:504] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:07.391160 28593 leader_election.cc:288] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:07.391458 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8752733ffcbd4667b586366c09d85e4f" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:07.391588 24951 raft_consensus.cc:3037] T 8752733ffcbd4667b586366c09d85e4f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:07.391722 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8752733ffcbd4667b586366c09d85e4f" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:07.391829 25083 raft_consensus.cc:3037] T 8752733ffcbd4667b586366c09d85e4f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:07.392629 24951 raft_consensus.cc:2454] T 8752733ffcbd4667b586366c09d85e4f P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:07.392765 25083 raft_consensus.cc:2454] T 8752733ffcbd4667b586366c09d85e4f P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:07.393083 25151 leader_election.cc:302] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:07.393415 28593 raft_consensus.cc:2781] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:07.393522 28593 raft_consensus.cc:686] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:07.393682 28593 consensus_queue.cc:237] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:07.395890 22223 catalog_manager.cc:5484] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:07.403067 22223 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:07.407946 29218 tablet.cc:1722] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:07.408836 29231 tablet.cc:1722] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:07.409435 29219 tablet.cc:1722] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:07.414400 22223 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=a85bcb9494e044348f00e0aff1f0cbb1] alter complete (version 3)
I20240502 07:59:07.519656 22229 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:07.519959 22229 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:07.525524 22229 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet e84280470d9c4047bbc471aac904dcd2
I20240502 07:59:07.526104 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet e84280470d9c4047bbc471aac904dcd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:07 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:07.526165 22743 tablet_service.cc:1514] Processing DeleteTablet for tablet e84280470d9c4047bbc471aac904dcd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:07 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:07.526396 29162 tablet_replica.cc:330] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:07.526561 29162 raft_consensus.cc:2229] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:07.526546 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet e84280470d9c4047bbc471aac904dcd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:07 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:07.526729 29253 tablet_replica.cc:330] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:07.526793 29162 raft_consensus.cc:2258] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:07.526863 29253 raft_consensus.cc:2229] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:07.527196 29253 raft_consensus.cc:2258] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:07.527357 29296 tablet_replica.cc:330] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:07.527624 29296 raft_consensus.cc:2229] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:07.527760 29162 ts_tablet_manager.cc:1875] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:07.527770 29296 raft_consensus.cc:2258] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:07.527835 29253 ts_tablet_manager.cc:1875] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:07.528430 29296 ts_tablet_manager.cc:1875] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:07.530593 29162 ts_tablet_manager.cc:1888] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:07.530707 29162 log.cc:1196] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/e84280470d9c4047bbc471aac904dcd2
I20240502 07:59:07.530797 29296 ts_tablet_manager.cc:1888] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:07.530874 29296 log.cc:1196] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/e84280470d9c4047bbc471aac904dcd2
I20240502 07:59:07.531039 29162 ts_tablet_manager.cc:1909] T e84280470d9c4047bbc471aac904dcd2 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:59:07.531076 29253 ts_tablet_manager.cc:1888] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:07.531195 29296 ts_tablet_manager.cc:1909] T e84280470d9c4047bbc471aac904dcd2 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:59:07.531796 29253 log.cc:1196] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/e84280470d9c4047bbc471aac904dcd2
I20240502 07:59:07.531884 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet e84280470d9c4047bbc471aac904dcd2 (table rebalancer_dynamic_table_new_name [id=a85bcb9494e044348f00e0aff1f0cbb1]) successfully deleted
I20240502 07:59:07.532109 29253 ts_tablet_manager.cc:1909] T e84280470d9c4047bbc471aac904dcd2 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:59:07.532166 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet e84280470d9c4047bbc471aac904dcd2 (table rebalancer_dynamic_table_new_name [id=a85bcb9494e044348f00e0aff1f0cbb1]) successfully deleted
I20240502 07:59:07.532814 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet e84280470d9c4047bbc471aac904dcd2 (table rebalancer_dynamic_table_new_name [id=a85bcb9494e044348f00e0aff1f0cbb1]) successfully deleted
I20240502 07:59:07.595775 28851 raft_consensus.cc:482] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:07.595904 28851 raft_consensus.cc:504] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.596321 28851 leader_election.cc:288] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:07.596673 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cd6c14c7c8b4ad29f2107e94fd11acc" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:07.596814 25084 raft_consensus.cc:2454] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:07.596908 29153 raft_consensus.cc:482] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:07.597002 29153 raft_consensus.cc:504] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:07.597131 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cd6c14c7c8b4ad29f2107e94fd11acc" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:07.597261 25214 raft_consensus.cc:2454] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:07.597432 29153 leader_election.cc:288] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:07.597608 24754 leader_election.cc:302] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:07.597775 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "351f047b9b53400b8cc6286359edb813" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:07.597801 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "351f047b9b53400b8cc6286359edb813" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:07.597890 25084 raft_consensus.cc:2454] T 351f047b9b53400b8cc6286359edb813 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:07.597913 28851 raft_consensus.cc:2781] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:07.597925 24951 raft_consensus.cc:2454] T 351f047b9b53400b8cc6286359edb813 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:07.597975 28851 raft_consensus.cc:482] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:07.598024 28851 raft_consensus.cc:3037] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:07.598160 22698 leader_election.cc:302] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:07.598392 29153 raft_consensus.cc:2781] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:07.598453 29153 raft_consensus.cc:482] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:07.598487 29153 raft_consensus.cc:3037] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:07.599157 28851 raft_consensus.cc:504] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.599501 29153 raft_consensus.cc:504] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:07.599565 28851 leader_election.cc:288] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:07.599857 29153 leader_election.cc:288] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:07.599870 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cd6c14c7c8b4ad29f2107e94fd11acc" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:07.599872 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cd6c14c7c8b4ad29f2107e94fd11acc" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:07.599999 25084 raft_consensus.cc:3037] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:07.600006 25214 raft_consensus.cc:3037] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:07.600076 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "351f047b9b53400b8cc6286359edb813" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:07.600176 24951 raft_consensus.cc:3037] T 351f047b9b53400b8cc6286359edb813 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:07.600284 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "351f047b9b53400b8cc6286359edb813" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:07.600394 25083 raft_consensus.cc:3037] T 351f047b9b53400b8cc6286359edb813 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:07.600976 25084 raft_consensus.cc:2454] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:07.600977 25214 raft_consensus.cc:2454] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:07.601132 24951 raft_consensus.cc:2454] T 351f047b9b53400b8cc6286359edb813 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:07.601207 25083 raft_consensus.cc:2454] T 351f047b9b53400b8cc6286359edb813 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:07.601305 24757 leader_election.cc:302] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:07.601598 28851 raft_consensus.cc:2781] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:07.601687 22701 leader_election.cc:302] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:07.601764 28851 raft_consensus.cc:686] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:59:07.601912 28851 consensus_queue.cc:237] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.603811 22229 catalog_manager.cc:5484] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:07.605594 29153 raft_consensus.cc:2781] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:07.605696 29153 raft_consensus.cc:686] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:59:07.605844 29153 consensus_queue.cc:237] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:07.607730 22229 catalog_manager.cc:5484] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:07.628466 22229 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:07.635628 25062 tablet_service.cc:1467] Processing CreateTablet for tablet c66bfe8f6acb4135abeb439bf4f33fd0 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=cd6eaccdb9b14e618fcb3b33ed8b5c0c]), partition=
I20240502 07:59:07.635919 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c66bfe8f6acb4135abeb439bf4f33fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:07.638475 22614 tablet_service.cc:1467] Processing CreateTablet for tablet c66bfe8f6acb4135abeb439bf4f33fd0 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=cd6eaccdb9b14e618fcb3b33ed8b5c0c]), partition=
I20240502 07:59:07.638525 24800 tablet_service.cc:1467] Processing CreateTablet for tablet c66bfe8f6acb4135abeb439bf4f33fd0 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=cd6eaccdb9b14e618fcb3b33ed8b5c0c]), partition=
I20240502 07:59:07.638715 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c66bfe8f6acb4135abeb439bf4f33fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:07.638744 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c66bfe8f6acb4135abeb439bf4f33fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:07.640586 29224 tablet_bootstrap.cc:492] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:07.641566 29224 tablet_bootstrap.cc:654] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:07.642472 29222 tablet_bootstrap.cc:492] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:07.643429 29222 tablet_bootstrap.cc:654] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:07.643589 25062 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:07.644760 29311 tablet_bootstrap.cc:492] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:07.644842 22351 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:07.647768 29224 tablet_bootstrap.cc:492] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:07.647868 29224 ts_tablet_manager.cc:1367] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:07.648517 29224 raft_consensus.cc:348] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.648644 29224 raft_consensus.cc:374] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:07.648679 29224 raft_consensus.cc:729] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:07.648800 29224 consensus_queue.cc:260] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.649109 29224 ts_tablet_manager.cc:1398] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:07.658412 24931 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:07.658874 24800 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:07.660627 22614 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:07.662153 29311 tablet_bootstrap.cc:654] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:07.663285 29222 tablet_bootstrap.cc:492] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:07.663275 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:07.663381 29222 ts_tablet_manager.cc:1367] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 07:59:07.663740 25193 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:07.663857 29222 raft_consensus.cc:348] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.663972 29222 raft_consensus.cc:374] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:07.664002 29222 raft_consensus.cc:729] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:07.664122 29222 consensus_queue.cc:260] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.665740 29222 ts_tablet_manager.cc:1398] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 07:59:07.677979 29311 tablet_bootstrap.cc:492] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:07.678114 29311 ts_tablet_manager.cc:1367] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20240502 07:59:07.678634 29311 raft_consensus.cc:348] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.678748 29311 raft_consensus.cc:374] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:07.678776 29311 raft_consensus.cc:729] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:07.678895 29311 consensus_queue.cc:260] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.679165 29311 ts_tablet_manager.cc:1398] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:07.689610 22743 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:07.721518 29034 raft_consensus.cc:482] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:07.721684 29034 raft_consensus.cc:504] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.722149 29034 leader_election.cc:288] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:07.723801 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c66bfe8f6acb4135abeb439bf4f33fd0" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:07.723966 24821 raft_consensus.cc:2454] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:07.724362 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c66bfe8f6acb4135abeb439bf4f33fd0" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:07.724510 22632 raft_consensus.cc:2454] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:07.725005 25019 leader_election.cc:302] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:07.725171 28464 raft_consensus.cc:2781] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:07.725251 28464 raft_consensus.cc:482] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:07.725306 28464 raft_consensus.cc:3037] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:07.726202 28464 raft_consensus.cc:504] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.726527 28464 leader_election.cc:288] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:07.726938 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c66bfe8f6acb4135abeb439bf4f33fd0" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:07.727068 24821 raft_consensus.cc:3037] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:07.727624 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c66bfe8f6acb4135abeb439bf4f33fd0" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:07.727733 22632 raft_consensus.cc:3037] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:07.728047 24821 raft_consensus.cc:2454] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:07.728713 22632 raft_consensus.cc:2454] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:07.729218 25019 leader_election.cc:302] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:07.729595 28464 raft_consensus.cc:2781] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:07.729682 28464 raft_consensus.cc:686] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:07.729830 28464 consensus_queue.cc:237] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:07.732170 22224 catalog_manager.cc:5484] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:07.755470 24951 consensus_queue.cc:237] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:07.756791 22501 raft_consensus.cc:1261] T 288b33b7a5f84af59bde5fae34cbe924 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:07.756778 22372 raft_consensus.cc:1261] T 288b33b7a5f84af59bde5fae34cbe924 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:07.757164 28686 consensus_queue.cc:1035] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:07.757378 29113 consensus_queue.cc:1035] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:07.759112 28815 raft_consensus.cc:2932] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:07.759646 22501 raft_consensus.cc:2932] T 288b33b7a5f84af59bde5fae34cbe924 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:07.759646 22372 raft_consensus.cc:2932] T 288b33b7a5f84af59bde5fae34cbe924 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:07.762324 22224 catalog_manager.cc:5484] T 288b33b7a5f84af59bde5fae34cbe924 P 1aecef81733c47b3b468504934357231 reported cstate change: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:07.764902 25062 tablet_service.cc:1514] Processing DeleteTablet for tablet 288b33b7a5f84af59bde5fae34cbe924 with delete_type TABLET_DATA_TOMBSTONED (TS 227eec4b5f014d349661f6e0b39b9055 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:07.765473 29334 tablet_replica.cc:330] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:07.765645 29334 raft_consensus.cc:2229] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:07.765775 29334 raft_consensus.cc:2258] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:07.766220 29334 ts_tablet_manager.cc:1875] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:07.768527 29334 ts_tablet_manager.cc:1888] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:07.768610 29334 log.cc:1196] T 288b33b7a5f84af59bde5fae34cbe924 P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/288b33b7a5f84af59bde5fae34cbe924
I20240502 07:59:07.769132 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet 288b33b7a5f84af59bde5fae34cbe924 (table rebalancer_extra_table_6 [id=ba79b61566954faab0f2bc52c0c0ba84]) successfully deleted
I20240502 07:59:07.814144 28593 consensus_queue.cc:1035] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:07.819006 29379 consensus_queue.cc:1035] T 8752733ffcbd4667b586366c09d85e4f P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:07.851119 22228 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:07.856526 24821 raft_consensus.cc:1261] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:07.856526 22632 raft_consensus.cc:1261] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:07.857007 29034 consensus_queue.cc:1035] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:07.857384 29034 consensus_queue.cc:1035] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:07.860703 22228 catalog_manager.cc:5576] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661) does not have the latest schema for tablet c66bfe8f6acb4135abeb439bf4f33fd0 (table rebalancer_dynamic_table [id=cd6eaccdb9b14e618fcb3b33ed8b5c0c]). Expected version 1 got 0
I20240502 07:59:07.864010 24800 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:07.864357 25062 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:07.871060 22224 catalog_manager.cc:5576] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703) does not have the latest schema for tablet c66bfe8f6acb4135abeb439bf4f33fd0 (table rebalancer_dynamic_table [id=cd6eaccdb9b14e618fcb3b33ed8b5c0c]). Expected version 1 got 0
I20240502 07:59:07.872251 24931 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:07.878494 29396 mvcc.cc:205] Tried to move back new op lower bound from 7023152119214977024 to 7023152118702989312. Current Snapshot: MvccSnapshot[applied={T|T < 7023152119214977024}]
I20240502 07:59:07.878729 22224 catalog_manager.cc:5576] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703) does not have the latest schema for tablet c66bfe8f6acb4135abeb439bf4f33fd0 (table rebalancer_dynamic_table [id=cd6eaccdb9b14e618fcb3b33ed8b5c0c]). Expected version 1 got 0
I20240502 07:59:07.880203 29399 tablet.cc:1722] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:07.880792 29399 tablet.cc:1620] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
I20240502 07:59:07.882612 22743 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:07.884518 29400 tablet.cc:1722] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:07.884876 29400 tablet.cc:1620] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:07.885424 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:07.887082 29398 tablet.cc:1722] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:07.887395 29398 tablet.cc:1620] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:07.888816 25193 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:07.898973 22614 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:07.900955 22224 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=cd6eaccdb9b14e618fcb3b33ed8b5c0c] alter complete (version 1)
I20240502 07:59:07.928470 22351 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:07.970520 22632 consensus_queue.cc:237] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:07.971694 22501 raft_consensus.cc:1261] T 17421eec61584fbcac13f441922269bc P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:07.971709 25083 raft_consensus.cc:1261] T 17421eec61584fbcac13f441922269bc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:07.972074 27965 consensus_queue.cc:1035] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:07.972291 27813 consensus_queue.cc:1035] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:07.974278 29346 raft_consensus.cc:2932] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:07.974871 22501 raft_consensus.cc:2932] T 17421eec61584fbcac13f441922269bc P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:07.974897 25083 raft_consensus.cc:2932] T 17421eec61584fbcac13f441922269bc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:07.978183 22223 catalog_manager.cc:5484] T 17421eec61584fbcac13f441922269bc P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index 4 to 5, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:07.981451 22743 tablet_service.cc:1514] Processing DeleteTablet for tablet 17421eec61584fbcac13f441922269bc with delete_type TABLET_DATA_TOMBSTONED (TS f00f3a1876444ad2bf9c919068ab8014 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:07.981741 29162 tablet_replica.cc:330] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:07.981906 29162 raft_consensus.cc:2229] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:07.982079 29162 raft_consensus.cc:2258] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:07.982647 29162 ts_tablet_manager.cc:1875] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:07.984974 29162 ts_tablet_manager.cc:1888] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:07.985035 29162 log.cc:1196] T 17421eec61584fbcac13f441922269bc P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/17421eec61584fbcac13f441922269bc
I20240502 07:59:07.985561 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 17421eec61584fbcac13f441922269bc (table rebalance_test_table_13 [id=490b2fcdef13484180791ae6a0d1907d]) successfully deleted
I20240502 07:59:07.997948 28851 consensus_queue.cc:1035] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:08.001639 29297 consensus_queue.cc:1035] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:08.006073 22228 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:08.011443 29398 tablet.cc:1722] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:08.011852 29398 tablet.cc:1620] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:08.012969 29400 tablet.cc:1722] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:08.013267 29400 tablet.cc:1620] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:08.013248 29399 tablet.cc:1722] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:08.013582 29399 tablet.cc:1620] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
I20240502 07:59:08.017467 29153 consensus_queue.cc:1035] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:08.018503 22228 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=cd6eaccdb9b14e618fcb3b33ed8b5c0c] alter complete (version 2)
I20240502 07:59:08.021550 28773 consensus_queue.cc:1035] T 351f047b9b53400b8cc6286359edb813 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:08.031972 22372 consensus_queue.cc:237] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.033640 22632 raft_consensus.cc:1261] T 15f70916ad6248e2a24334020df2c55e P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:08.033749 22305 consensus_peers.cc:485] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): Couldn't send request to peer 227eec4b5f014d349661f6e0b39b9055. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 15f70916ad6248e2a24334020df2c55e. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:08.034075 29456 consensus_queue.cc:1035] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:08.034955 22501 raft_consensus.cc:1261] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:08.035337 29456 consensus_queue.cc:1035] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:08.037118 29455 raft_consensus.cc:2932] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:08.037525 22501 raft_consensus.cc:2932] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:08.039780 22632 raft_consensus.cc:2932] T 15f70916ad6248e2a24334020df2c55e P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:08.040562 22224 catalog_manager.cc:5484] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:08.063238 24952 consensus_queue.cc:237] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:08.064561 25215 raft_consensus.cc:1261] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:08.064716 24888 consensus_peers.cc:485] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 -> Peer f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): Couldn't send request to peer f00f3a1876444ad2bf9c919068ab8014. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c651077242584f57a165073de3e4a088. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:08.064926 25083 raft_consensus.cc:1261] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:08.065286 28686 consensus_queue.cc:1035] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:08.065521 29113 consensus_queue.cc:1035] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:08.067078 29113 raft_consensus.cc:2932] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:08.067571 25215 raft_consensus.cc:2932] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:08.068854 25083 raft_consensus.cc:2932] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:08.070788 22223 catalog_manager.cc:5484] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:08.122028 22222 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:08.129624 29398 tablet.cc:1722] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:08.130556 29400 tablet.cc:1722] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:08.130743 29399 tablet.cc:1722] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:08.137799 22225 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=cd6eaccdb9b14e618fcb3b33ed8b5c0c] alter complete (version 3)
I20240502 07:59:08.168166 24800 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:08.169528 24931 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:08.173497 22743 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:08.182121 25062 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:08.182482 22225 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:08.185030 25193 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:08.190454 24800 tablet_service.cc:1467] Processing CreateTablet for tablet bf443ada59854e6da532aa5832a9a564 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=a01ec1f3fc0149c7a0e9a1274c6bd139]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:08.190785 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf443ada59854e6da532aa5832a9a564. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:08.190858 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 3d1511fc27c24820afaa6e41161be8bb (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=a01ec1f3fc0149c7a0e9a1274c6bd139]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:08.191109 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d1511fc27c24820afaa6e41161be8bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:08.193383 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 6312bc58373346e384c9ebd223e1612b (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=a01ec1f3fc0149c7a0e9a1274c6bd139]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:08.193648 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6312bc58373346e384c9ebd223e1612b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:08.194379 22743 tablet_service.cc:1467] Processing CreateTablet for tablet bf443ada59854e6da532aa5832a9a564 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=a01ec1f3fc0149c7a0e9a1274c6bd139]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:08.194634 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf443ada59854e6da532aa5832a9a564. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:08.195941 24931 tablet_service.cc:1467] Processing CreateTablet for tablet 6312bc58373346e384c9ebd223e1612b (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=a01ec1f3fc0149c7a0e9a1274c6bd139]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:08.196183 24931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6312bc58373346e384c9ebd223e1612b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:08.197985 25194 tablet_service.cc:1467] Processing CreateTablet for tablet 3d1511fc27c24820afaa6e41161be8bb (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=a01ec1f3fc0149c7a0e9a1274c6bd139]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:08.198276 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d1511fc27c24820afaa6e41161be8bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:08.198524 25062 tablet_service.cc:1467] Processing CreateTablet for tablet bf443ada59854e6da532aa5832a9a564 (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=a01ec1f3fc0149c7a0e9a1274c6bd139]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:08.198750 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf443ada59854e6da532aa5832a9a564. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:08.197985 25193 tablet_service.cc:1467] Processing CreateTablet for tablet 6312bc58373346e384c9ebd223e1612b (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=a01ec1f3fc0149c7a0e9a1274c6bd139]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:08.200675 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6312bc58373346e384c9ebd223e1612b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:08.200969 29535 tablet_bootstrap.cc:492] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:08.201421 29533 tablet_bootstrap.cc:492] T 6312bc58373346e384c9ebd223e1612b P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:08.203514 29533 tablet_bootstrap.cc:654] T 6312bc58373346e384c9ebd223e1612b P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:08.203727 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 3d1511fc27c24820afaa6e41161be8bb (DEFAULT_TABLE table=rebalancer_extra_table_30 [id=a01ec1f3fc0149c7a0e9a1274c6bd139]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:08.203986 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d1511fc27c24820afaa6e41161be8bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:08.204047 29535 tablet_bootstrap.cc:654] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:08.205250 29532 tablet_bootstrap.cc:492] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:08.206243 29532 tablet_bootstrap.cc:654] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:08.207819 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:08.209369 29535 tablet_bootstrap.cc:492] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:08.209507 29535 ts_tablet_manager.cc:1367] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20240502 07:59:08.209985 29535 raft_consensus.cc:348] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.210196 29535 raft_consensus.cc:374] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:08.210265 29535 raft_consensus.cc:729] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:08.210456 29535 consensus_queue.cc:260] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.210705 29531 tablet_bootstrap.cc:492] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:08.210883 29535 ts_tablet_manager.cc:1398] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:08.210981 29536 tablet_bootstrap.cc:492] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:08.211567 29533 tablet_bootstrap.cc:492] T 6312bc58373346e384c9ebd223e1612b P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:08.211706 29533 ts_tablet_manager.cc:1367] T 6312bc58373346e384c9ebd223e1612b P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20240502 07:59:08.212167 29533 raft_consensus.cc:348] T 6312bc58373346e384c9ebd223e1612b P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.212324 29533 raft_consensus.cc:374] T 6312bc58373346e384c9ebd223e1612b P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:08.212392 29533 raft_consensus.cc:729] T 6312bc58373346e384c9ebd223e1612b P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:08.212563 29533 consensus_queue.cc:260] T 6312bc58373346e384c9ebd223e1612b P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.212968 29533 ts_tablet_manager.cc:1398] T 6312bc58373346e384c9ebd223e1612b P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:08.213171 29533 tablet_bootstrap.cc:492] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:08.214095 29533 tablet_bootstrap.cc:654] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:08.215025 29536 tablet_bootstrap.cc:654] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:08.215363 29531 tablet_bootstrap.cc:654] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:08.215905 29532 tablet_bootstrap.cc:492] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:08.216037 29532 ts_tablet_manager.cc:1367] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 07:59:08.216596 29532 raft_consensus.cc:348] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:08.216750 29532 raft_consensus.cc:374] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:08.216820 29532 raft_consensus.cc:729] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:08.216892 29536 tablet_bootstrap.cc:492] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:08.217027 29536 ts_tablet_manager.cc:1367] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:08.216996 29532 consensus_queue.cc:260] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:08.217928 29532 ts_tablet_manager.cc:1398] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:08.218642 29533 tablet_bootstrap.cc:492] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:08.218621 22351 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:08.218587 29536 raft_consensus.cc:348] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:08.218689 29536 raft_consensus.cc:374] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:08.218770 29536 raft_consensus.cc:729] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:08.218883 29533 ts_tablet_manager.cc:1367] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:08.218961 29536 consensus_queue.cc:260] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:08.219400 29533 raft_consensus.cc:348] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.219570 29533 raft_consensus.cc:374] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:08.219642 29533 raft_consensus.cc:729] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:08.219811 29533 consensus_queue.cc:260] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.219941 29536 ts_tablet_manager.cc:1398] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:08.220186 29536 tablet_bootstrap.cc:492] T 6312bc58373346e384c9ebd223e1612b P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:08.220208 29533 ts_tablet_manager.cc:1398] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:08.221171 29536 tablet_bootstrap.cc:654] T 6312bc58373346e384c9ebd223e1612b P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:08.221172 29534 tablet_bootstrap.cc:492] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:08.222468 29534 tablet_bootstrap.cc:654] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:08.228001 29534 tablet_bootstrap.cc:492] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:08.228150 29534 ts_tablet_manager.cc:1367] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:08.228631 29534 raft_consensus.cc:348] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.228848 29534 raft_consensus.cc:374] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:08.228951 29534 raft_consensus.cc:729] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:08.229307 29534 consensus_queue.cc:260] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.230629 29534 ts_tablet_manager.cc:1398] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:08.236147 29531 tablet_bootstrap.cc:492] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:08.236318 29531 ts_tablet_manager.cc:1367] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20240502 07:59:08.237246 29531 raft_consensus.cc:348] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.238471 29531 raft_consensus.cc:374] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:08.238562 29531 raft_consensus.cc:729] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:08.238754 29531 consensus_queue.cc:260] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.239796 22614 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:08.241822 29536 tablet_bootstrap.cc:492] T 6312bc58373346e384c9ebd223e1612b P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:08.241914 29536 ts_tablet_manager.cc:1367] T 6312bc58373346e384c9ebd223e1612b P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20240502 07:59:08.242409 29536 raft_consensus.cc:348] T 6312bc58373346e384c9ebd223e1612b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.242516 29536 raft_consensus.cc:374] T 6312bc58373346e384c9ebd223e1612b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:08.242542 29536 raft_consensus.cc:729] T 6312bc58373346e384c9ebd223e1612b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:08.242666 29536 consensus_queue.cc:260] T 6312bc58373346e384c9ebd223e1612b P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.244174 22230 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:08.244307 22230 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:08.244311 29536 ts_tablet_manager.cc:1398] T 6312bc58373346e384c9ebd223e1612b P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:08.245863 29531 ts_tablet_manager.cc:1398] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20240502 07:59:08.247478 29531 tablet_bootstrap.cc:492] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:08.250075 22230 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet c66bfe8f6acb4135abeb439bf4f33fd0
I20240502 07:59:08.251015 24800 tablet_service.cc:1514] Processing DeleteTablet for tablet c66bfe8f6acb4135abeb439bf4f33fd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:08 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:59:08.251230 29531 tablet_bootstrap.cc:654] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:08.252221 22614 tablet_service.cc:1514] Processing DeleteTablet for tablet c66bfe8f6acb4135abeb439bf4f33fd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:08 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:08.253517 25062 tablet_service.cc:1514] Processing DeleteTablet for tablet c66bfe8f6acb4135abeb439bf4f33fd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:08 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:08.253700 29334 tablet_replica.cc:330] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:08.253847 29334 raft_consensus.cc:2229] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:08.254048 29334 raft_consensus.cc:2258] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:08.254690 29334 ts_tablet_manager.cc:1875] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:08.256834 29334 ts_tablet_manager.cc:1888] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:08.256902 29334 log.cc:1196] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/c66bfe8f6acb4135abeb439bf4f33fd0
I20240502 07:59:08.257169 29334 ts_tablet_manager.cc:1909] T c66bfe8f6acb4135abeb439bf4f33fd0 P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:59:08.258363 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet c66bfe8f6acb4135abeb439bf4f33fd0 (table rebalancer_dynamic_table_new_name [id=cd6eaccdb9b14e618fcb3b33ed8b5c0c]) successfully deleted
I20240502 07:59:08.259174 29531 tablet_bootstrap.cc:492] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:08.259279 29531 ts_tablet_manager.cc:1367] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20240502 07:59:08.259738 29531 raft_consensus.cc:348] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:08.259837 29531 raft_consensus.cc:374] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:08.259864 29531 raft_consensus.cc:729] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:08.259976 29531 consensus_queue.cc:260] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:08.260222 29531 ts_tablet_manager.cc:1398] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:08.261273 29545 tablet_replica.cc:330] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:59:08.261422 29545 raft_consensus.cc:2229] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:08.261552 29545 raft_consensus.cc:2258] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:08.264250 29547 tablet_replica.cc:330] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:08.264395 29547 raft_consensus.cc:2229] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:08.264513 29547 raft_consensus.cc:2258] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:08.267030 29547 ts_tablet_manager.cc:1875] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:08.269155 29545 ts_tablet_manager.cc:1875] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:08.271489 29545 ts_tablet_manager.cc:1888] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:08.271602 29545 log.cc:1196] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/c66bfe8f6acb4135abeb439bf4f33fd0
I20240502 07:59:08.271898 29545 ts_tablet_manager.cc:1909] T c66bfe8f6acb4135abeb439bf4f33fd0 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:59:08.272794 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet c66bfe8f6acb4135abeb439bf4f33fd0 (table rebalancer_dynamic_table_new_name [id=cd6eaccdb9b14e618fcb3b33ed8b5c0c]) successfully deleted
I20240502 07:59:08.274278 29547 ts_tablet_manager.cc:1888] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:08.274364 29547 log.cc:1196] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/c66bfe8f6acb4135abeb439bf4f33fd0
I20240502 07:59:08.274708 29547 ts_tablet_manager.cc:1909] T c66bfe8f6acb4135abeb439bf4f33fd0 P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:59:08.275400 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet c66bfe8f6acb4135abeb439bf4f33fd0 (table rebalancer_dynamic_table_new_name [id=cd6eaccdb9b14e618fcb3b33ed8b5c0c]) successfully deleted
I20240502 07:59:08.315913 28815 raft_consensus.cc:482] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:08.316051 28815 raft_consensus.cc:504] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.316507 28815 leader_election.cc:288] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:08.316870 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6312bc58373346e384c9ebd223e1612b" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:08.317006 25215 raft_consensus.cc:2454] T 6312bc58373346e384c9ebd223e1612b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:08.316977 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6312bc58373346e384c9ebd223e1612b" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:08.317126 25083 raft_consensus.cc:2454] T 6312bc58373346e384c9ebd223e1612b P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:08.317395 24888 leader_election.cc:302] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:08.317641 28815 raft_consensus.cc:2781] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:08.317694 28815 raft_consensus.cc:482] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:08.317744 28815 raft_consensus.cc:3037] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:08.318790 28815 raft_consensus.cc:504] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.319185 28815 leader_election.cc:288] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:08.319455 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6312bc58373346e384c9ebd223e1612b" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:08.319579 25215 raft_consensus.cc:3037] T 6312bc58373346e384c9ebd223e1612b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:08.319653 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6312bc58373346e384c9ebd223e1612b" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:08.319775 25083 raft_consensus.cc:3037] T 6312bc58373346e384c9ebd223e1612b P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:08.320546 25215 raft_consensus.cc:2454] T 6312bc58373346e384c9ebd223e1612b P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:08.320624 25083 raft_consensus.cc:2454] T 6312bc58373346e384c9ebd223e1612b P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:08.320832 24888 leader_election.cc:302] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:08.321017 28815 raft_consensus.cc:2781] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:08.321086 28815 raft_consensus.cc:686] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:08.321211 28815 consensus_queue.cc:237] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.324163 22223 catalog_manager.cc:5484] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:08.351938 22221 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:08.358749 24931 tablet_service.cc:1467] Processing CreateTablet for tablet 32e8fea8c9dd4fd88525f8d2d0339970 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=56c1f4ae38dc43749ae3fd1e2fabe2eb]), partition=
I20240502 07:59:08.358880 25193 tablet_service.cc:1467] Processing CreateTablet for tablet 32e8fea8c9dd4fd88525f8d2d0339970 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=56c1f4ae38dc43749ae3fd1e2fabe2eb]), partition=
I20240502 07:59:08.359035 24931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32e8fea8c9dd4fd88525f8d2d0339970. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:08.359128 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32e8fea8c9dd4fd88525f8d2d0339970. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:08.360944 29536 tablet_bootstrap.cc:492] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:08.361222 29535 tablet_bootstrap.cc:492] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:08.361874 29536 tablet_bootstrap.cc:654] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:08.362118 29535 tablet_bootstrap.cc:654] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:08.363261 25062 tablet_service.cc:1467] Processing CreateTablet for tablet 32e8fea8c9dd4fd88525f8d2d0339970 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=56c1f4ae38dc43749ae3fd1e2fabe2eb]), partition=
I20240502 07:59:08.363490 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32e8fea8c9dd4fd88525f8d2d0339970. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:08.363546 29536 tablet_bootstrap.cc:492] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:08.363636 29536 ts_tablet_manager.cc:1367] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 07:59:08.364076 29536 raft_consensus.cc:348] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:08.364178 29536 raft_consensus.cc:374] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:08.364212 29536 raft_consensus.cc:729] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:08.364323 29536 consensus_queue.cc:260] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:08.364620 29536 ts_tablet_manager.cc:1398] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:08.365617 29533 tablet_bootstrap.cc:492] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:08.366475 29533 tablet_bootstrap.cc:654] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:08.368697 29535 tablet_bootstrap.cc:492] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:08.368814 29535 ts_tablet_manager.cc:1367] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:59:08.369254 29535 raft_consensus.cc:348] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:08.369396 29535 raft_consensus.cc:374] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:08.369432 29535 raft_consensus.cc:729] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:08.369565 29535 consensus_queue.cc:260] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:08.369894 29535 ts_tablet_manager.cc:1398] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:08.370057 29533 tablet_bootstrap.cc:492] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:08.370128 29533 ts_tablet_manager.cc:1367] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20240502 07:59:08.370636 29533 raft_consensus.cc:348] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:08.370761 29533 raft_consensus.cc:374] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:08.370796 29533 raft_consensus.cc:729] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:08.370903 29533 consensus_queue.cc:260] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:08.371198 29533 ts_tablet_manager.cc:1398] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:08.383852 28851 raft_consensus.cc:482] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:08.383960 28851 raft_consensus.cc:504] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.384335 28851 leader_election.cc:288] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:08.384689 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf443ada59854e6da532aa5832a9a564" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:08.384713 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf443ada59854e6da532aa5832a9a564" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:08.384826 25083 raft_consensus.cc:2454] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:08.384832 22764 raft_consensus.cc:2454] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:08.385474 24754 leader_election.cc:302] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:08.385663 28851 raft_consensus.cc:2781] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:08.385732 28851 raft_consensus.cc:482] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:08.385774 28851 raft_consensus.cc:3037] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:08.386716 28851 raft_consensus.cc:504] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.387069 28851 leader_election.cc:288] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:08.387418 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf443ada59854e6da532aa5832a9a564" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:08.387411 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf443ada59854e6da532aa5832a9a564" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:08.387537 25083 raft_consensus.cc:3037] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:08.387591 22764 raft_consensus.cc:3037] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:08.388542 25083 raft_consensus.cc:2454] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:08.388690 22764 raft_consensus.cc:2454] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:08.388875 24754 leader_election.cc:302] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:08.389158 28851 raft_consensus.cc:2781] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:08.389262 28851 raft_consensus.cc:686] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:59:08.389426 28851 consensus_queue.cc:237] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.391305 22223 catalog_manager.cc:5484] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:08.452294 29596 ts_tablet_manager.cc:902] T 15f70916ad6248e2a24334020df2c55e P 227eec4b5f014d349661f6e0b39b9055: Initiating tablet copy from peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:08.452879 29596 tablet_copy_client.cc:308] T 15f70916ad6248e2a24334020df2c55e P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.129:38535
I20240502 07:59:08.453385 22392 tablet_copy_service.cc:140] P b533a7d8d97943609c6e28e3e36b78a4: Received BeginTabletCopySession request for tablet 15f70916ad6248e2a24334020df2c55e from peer 227eec4b5f014d349661f6e0b39b9055 ({username='slave'} at 127.15.129.135:41433)
I20240502 07:59:08.453500 22392 tablet_copy_service.cc:161] P b533a7d8d97943609c6e28e3e36b78a4: Beginning new tablet copy session on tablet 15f70916ad6248e2a24334020df2c55e from peer 227eec4b5f014d349661f6e0b39b9055 at {username='slave'} at 127.15.129.135:41433: session id = 227eec4b5f014d349661f6e0b39b9055-15f70916ad6248e2a24334020df2c55e
I20240502 07:59:08.454407 22392 tablet_copy_source_session.cc:205] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:08.455245 29596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15f70916ad6248e2a24334020df2c55e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:08.456982 29596 tablet_copy_client.cc:783] T 15f70916ad6248e2a24334020df2c55e P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:08.457127 29596 tablet_copy_client.cc:647] T 15f70916ad6248e2a24334020df2c55e P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:08.458415 29596 tablet_copy_client.cc:523] T 15f70916ad6248e2a24334020df2c55e P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:08.459488 29596 tablet_bootstrap.cc:492] T 15f70916ad6248e2a24334020df2c55e P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:08.482203 29596 tablet_bootstrap.cc:492] T 15f70916ad6248e2a24334020df2c55e P 227eec4b5f014d349661f6e0b39b9055: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:08.482568 29596 tablet_bootstrap.cc:492] T 15f70916ad6248e2a24334020df2c55e P 227eec4b5f014d349661f6e0b39b9055: Bootstrap complete.
I20240502 07:59:08.482762 29596 ts_tablet_manager.cc:1367] T 15f70916ad6248e2a24334020df2c55e P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.023s user 0.024s sys 0.000s
I20240502 07:59:08.483182 29596 raft_consensus.cc:348] T 15f70916ad6248e2a24334020df2c55e P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.483295 29596 raft_consensus.cc:729] T 15f70916ad6248e2a24334020df2c55e P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:08.483424 29596 consensus_queue.cc:260] T 15f70916ad6248e2a24334020df2c55e P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:08.483714 29596 ts_tablet_manager.cc:1398] T 15f70916ad6248e2a24334020df2c55e P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:08.484265 22392 tablet_copy_service.cc:342] P b533a7d8d97943609c6e28e3e36b78a4: Request end of tablet copy session 227eec4b5f014d349661f6e0b39b9055-15f70916ad6248e2a24334020df2c55e received from {username='slave'} at 127.15.129.135:41433
I20240502 07:59:08.484352 22392 tablet_copy_service.cc:433] P b533a7d8d97943609c6e28e3e36b78a4: ending tablet copy session 227eec4b5f014d349661f6e0b39b9055-15f70916ad6248e2a24334020df2c55e on tablet 15f70916ad6248e2a24334020df2c55e with peer 227eec4b5f014d349661f6e0b39b9055
I20240502 07:59:08.484586 29602 ts_tablet_manager.cc:902] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014: Initiating tablet copy from peer 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:08.485121 29602 tablet_copy_client.cc:308] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.134:35995
I20240502 07:59:08.485960 24972 tablet_copy_service.cc:140] P 1aecef81733c47b3b468504934357231: Received BeginTabletCopySession request for tablet c651077242584f57a165073de3e4a088 from peer f00f3a1876444ad2bf9c919068ab8014 ({username='slave'} at 127.15.129.132:59331)
I20240502 07:59:08.486047 24972 tablet_copy_service.cc:161] P 1aecef81733c47b3b468504934357231: Beginning new tablet copy session on tablet c651077242584f57a165073de3e4a088 from peer f00f3a1876444ad2bf9c919068ab8014 at {username='slave'} at 127.15.129.132:59331: session id = f00f3a1876444ad2bf9c919068ab8014-c651077242584f57a165073de3e4a088
I20240502 07:59:08.486907 24972 tablet_copy_source_session.cc:205] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:08.488162 29602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c651077242584f57a165073de3e4a088. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:08.490087 29602 tablet_copy_client.cc:783] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:08.490227 29602 tablet_copy_client.cc:647] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:08.491289 29602 tablet_copy_client.cc:523] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:08.492439 29602 tablet_bootstrap.cc:492] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:08.495435 29602 tablet_bootstrap.cc:492] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:08.495682 29602 tablet_bootstrap.cc:492] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap complete.
I20240502 07:59:08.495841 29602 ts_tablet_manager.cc:1367] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20240502 07:59:08.496254 29602 raft_consensus.cc:348] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:08.496369 29602 raft_consensus.cc:729] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:08.496505 29602 consensus_queue.cc:260] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:08.497267 29602 ts_tablet_manager.cc:1398] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:08.497720 24972 tablet_copy_service.cc:342] P 1aecef81733c47b3b468504934357231: Request end of tablet copy session f00f3a1876444ad2bf9c919068ab8014-c651077242584f57a165073de3e4a088 received from {username='slave'} at 127.15.129.132:59331
I20240502 07:59:08.497792 24972 tablet_copy_service.cc:433] P 1aecef81733c47b3b468504934357231: ending tablet copy session f00f3a1876444ad2bf9c919068ab8014-c651077242584f57a165073de3e4a088 on tablet c651077242584f57a165073de3e4a088 with peer f00f3a1876444ad2bf9c919068ab8014
I20240502 07:59:08.545703 28815 raft_consensus.cc:482] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:08.545827 28815 raft_consensus.cc:504] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:08.546191 28815 leader_election.cc:288] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:08.546501 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32e8fea8c9dd4fd88525f8d2d0339970" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:08.546535 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32e8fea8c9dd4fd88525f8d2d0339970" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:08.546626 25083 raft_consensus.cc:2454] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:08.546663 25215 raft_consensus.cc:2454] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:08.546895 24885 leader_election.cc:302] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
I20240502 07:59:08.547051 28815 raft_consensus.cc:2781] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:08.547101 28815 raft_consensus.cc:482] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:08.547127 28815 raft_consensus.cc:3037] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:08.547973 28815 raft_consensus.cc:504] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:08.548269 28815 leader_election.cc:288] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:08.548556 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32e8fea8c9dd4fd88525f8d2d0339970" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:08.548583 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32e8fea8c9dd4fd88525f8d2d0339970" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:08.548686 25215 raft_consensus.cc:3037] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:08.548666 25083 raft_consensus.cc:3037] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:08.549598 25215 raft_consensus.cc:2454] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:08.549805 25083 raft_consensus.cc:2454] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:08.549923 24888 leader_election.cc:302] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:08.550069 28815 raft_consensus.cc:2781] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:08.550158 28815 raft_consensus.cc:686] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:08.550303 28815 consensus_queue.cc:237] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:08.552291 22223 catalog_manager.cc:5484] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:08.570816 28851 raft_consensus.cc:482] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:08.570928 28851 raft_consensus.cc:504] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:08.571273 28851 leader_election.cc:288] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:08.571595 22499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d1511fc27c24820afaa6e41161be8bb" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:08.571635 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d1511fc27c24820afaa6e41161be8bb" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:08.571714 22499 raft_consensus.cc:2454] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:08.571789 25215 raft_consensus.cc:2454] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:08.572031 24754 leader_election.cc:302] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:08.572171 28851 raft_consensus.cc:2781] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:08.572232 28851 raft_consensus.cc:482] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:08.572270 28851 raft_consensus.cc:3037] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:08.573143 28851 raft_consensus.cc:504] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:08.573523 28851 leader_election.cc:288] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:08.573853 22499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d1511fc27c24820afaa6e41161be8bb" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:08.573863 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d1511fc27c24820afaa6e41161be8bb" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:08.573954 25215 raft_consensus.cc:3037] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:08.573961 22499 raft_consensus.cc:3037] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:08.574769 25215 raft_consensus.cc:2454] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:08.574894 22499 raft_consensus.cc:2454] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:08.575047 24757 leader_election.cc:302] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:08.575227 28851 raft_consensus.cc:2781] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:08.575306 28851 raft_consensus.cc:686] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:59:08.575446 28851 consensus_queue.cc:237] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:08.577313 22223 catalog_manager.cc:5484] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:08.601042 24800 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:08.607187 22351 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:08.607194 24931 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:08.611799 25062 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:08.615854 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:08.623028 22614 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:08.623931 25193 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:08.655505 22743 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:08.703352 22223 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:08.708012 25215 raft_consensus.cc:1261] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:08.708364 25084 raft_consensus.cc:1261] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:08.708742 28686 consensus_queue.cc:1035] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:08.708951 25083 consensus_queue.cc:237] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:08.709136 28686 consensus_queue.cc:1035] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:08.710287 22372 raft_consensus.cc:1261] T 9f44c905bfaa4db093a1b39b6c21e03c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:08.710343 24821 raft_consensus.cc:1261] T 9f44c905bfaa4db093a1b39b6c21e03c P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:08.710691 28464 consensus_queue.cc:1035] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:08.710888 29622 consensus_queue.cc:1035] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:08.712790 28464 raft_consensus.cc:2932] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:08.712957 22223 catalog_manager.cc:5576] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733) does not have the latest schema for tablet 32e8fea8c9dd4fd88525f8d2d0339970 (table rebalancer_dynamic_table [id=56c1f4ae38dc43749ae3fd1e2fabe2eb]). Expected version 1 got 0
I20240502 07:59:08.713197 24821 raft_consensus.cc:2932] T 9f44c905bfaa4db093a1b39b6c21e03c P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:08.714085 22372 raft_consensus.cc:2932] T 9f44c905bfaa4db093a1b39b6c21e03c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:08.717211 22222 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 32e8fea8c9dd4fd88525f8d2d0339970 (table rebalancer_dynamic_table [id=56c1f4ae38dc43749ae3fd1e2fabe2eb]). Expected version 1 got 0
I20240502 07:59:08.717610 22230 catalog_manager.cc:5484] T 9f44c905bfaa4db093a1b39b6c21e03c P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:08.722038 24931 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f44c905bfaa4db093a1b39b6c21e03c with delete_type TABLET_DATA_TOMBSTONED (TS 1aecef81733c47b3b468504934357231 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:08.722344 29651 tablet_replica.cc:330] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:08.722563 29651 raft_consensus.cc:2229] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:08.722752 29651 raft_consensus.cc:2258] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:08.723155 29651 ts_tablet_manager.cc:1875] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:08.724747 22223 catalog_manager.cc:5576] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733) does not have the latest schema for tablet 32e8fea8c9dd4fd88525f8d2d0339970 (table rebalancer_dynamic_table [id=56c1f4ae38dc43749ae3fd1e2fabe2eb]). Expected version 1 got 0
I20240502 07:59:08.725431 29651 ts_tablet_manager.cc:1888] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:08.725634 29652 tablet.cc:1722] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:08.726006 29652 tablet.cc:1620] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:08.726248 29651 log.cc:1196] T 9f44c905bfaa4db093a1b39b6c21e03c P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/9f44c905bfaa4db093a1b39b6c21e03c
I20240502 07:59:08.726848 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet 9f44c905bfaa4db093a1b39b6c21e03c (table rebalancer_extra_table_8 [id=72f8b22955164639b2802532262e4e2f]) successfully deleted
I20240502 07:59:08.731415 22223 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=56c1f4ae38dc43749ae3fd1e2fabe2eb] alter complete (version 1)
I20240502 07:59:08.733975 29650 tablet.cc:1722] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:08.734364 29650 tablet.cc:1620] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:08.738742 22223 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 32e8fea8c9dd4fd88525f8d2d0339970 (table rebalancer_dynamic_table [id=56c1f4ae38dc43749ae3fd1e2fabe2eb]). Expected version 1 got 0
I20240502 07:59:08.741631 29655 tablet.cc:1722] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:08.744493 29655 tablet.cc:1620] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:08.748220 28815 consensus_queue.cc:1035] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:08.751848 28686 consensus_queue.cc:1035] T 6312bc58373346e384c9ebd223e1612b P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:08.834687 25062 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:08.836898 22223 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:08.838579 24931 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:08.841869 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:08.842828 25193 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:08.846570 29650 tablet.cc:1722] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:08.846902 29650 tablet.cc:1620] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:08.855710 29655 tablet.cc:1722] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:08.856062 29655 tablet.cc:1620] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:08.858479 29652 tablet.cc:1722] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:08.858790 29652 tablet.cc:1620] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:08.858841 22228 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=56c1f4ae38dc43749ae3fd1e2fabe2eb] alter complete (version 2)
I20240502 07:59:08.862007 24800 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:08.869015 22743 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:08.869603 22351 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:08.885916 22764 raft_consensus.cc:1203] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:08.890903 28851 consensus_queue.cc:1035] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:08.892774 22614 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:08.896286 25084 raft_consensus.cc:1203] T 15f70916ad6248e2a24334020df2c55e P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:08.901540 29297 consensus_queue.cc:1035] T bf443ada59854e6da532aa5832a9a564 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:08.958051 22499 consensus_queue.cc:237] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:08.959623 22372 raft_consensus.cc:1261] T cc97164c8cfe4f91afabcc606d9821ef P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:08.959617 22632 raft_consensus.cc:1261] T cc97164c8cfe4f91afabcc606d9821ef P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:08.960054 29725 consensus_queue.cc:1035] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:08.960247 28757 consensus_queue.cc:1035] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:08.961802 22223 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:08.962184 28757 raft_consensus.cc:2932] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:08.962685 22632 raft_consensus.cc:2932] T cc97164c8cfe4f91afabcc606d9821ef P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:08.962744 22372 raft_consensus.cc:2932] T cc97164c8cfe4f91afabcc606d9821ef P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:08.966297 22228 catalog_manager.cc:5484] T cc97164c8cfe4f91afabcc606d9821ef P fabddb16c85944648a710ec52fc7244c reported cstate change: config changed from index 2 to 3, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) evicted. New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:08.970572 29650 tablet.cc:1722] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:08.972337 29652 tablet.cc:1722] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:08.973413 25193 tablet_service.cc:1514] Processing DeleteTablet for tablet cc97164c8cfe4f91afabcc606d9821ef with delete_type TABLET_DATA_TOMBSTONED (TS 43930dee0cd5498fa8e357a34dd71432 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:08.974328 29655 tablet.cc:1722] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:08.975493 29730 tablet_replica.cc:330] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:08.975697 29730 raft_consensus.cc:2229] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:08.976485 29730 raft_consensus.cc:2258] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:08.977262 29730 ts_tablet_manager.cc:1875] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:08.979385 22230 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=56c1f4ae38dc43749ae3fd1e2fabe2eb] alter complete (version 3)
I20240502 07:59:08.981591 29730 ts_tablet_manager.cc:1888] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:08.981676 29730 log.cc:1196] T cc97164c8cfe4f91afabcc606d9821ef P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/cc97164c8cfe4f91afabcc606d9821ef
I20240502 07:59:08.982216 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet cc97164c8cfe4f91afabcc606d9821ef (table rebalancer_extra_table_2 [id=5b6a061be2624cb1852eb43e3a4cb66b]) successfully deleted
I20240502 07:59:09.081490 22227 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:09.081665 22227 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:09.082412 25062 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:09.086455 22227 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 32e8fea8c9dd4fd88525f8d2d0339970
I20240502 07:59:09.087030 24800 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:09.087234 25193 tablet_service.cc:1514] Processing DeleteTablet for tablet 32e8fea8c9dd4fd88525f8d2d0339970 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:09 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:09.087623 29730 tablet_replica.cc:330] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:09.087767 29730 raft_consensus.cc:2229] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:09.087880 29730 raft_consensus.cc:2258] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:09.088943 25062 tablet_service.cc:1514] Processing DeleteTablet for tablet 32e8fea8c9dd4fd88525f8d2d0339970 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:09 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:09.089557 25193 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:09.092028 24931 tablet_service.cc:1514] Processing DeleteTablet for tablet 32e8fea8c9dd4fd88525f8d2d0339970 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:09 UTC) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:09.092262 29730 ts_tablet_manager.cc:1875] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:09.094427 29730 ts_tablet_manager.cc:1888] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:09.094507 29730 log.cc:1196] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/32e8fea8c9dd4fd88525f8d2d0339970
I20240502 07:59:09.094786 29730 ts_tablet_manager.cc:1909] T 32e8fea8c9dd4fd88525f8d2d0339970 P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:59:09.095532 29651 tablet_replica.cc:330] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:09.095592 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 32e8fea8c9dd4fd88525f8d2d0339970 (table rebalancer_dynamic_table_new_name [id=56c1f4ae38dc43749ae3fd1e2fabe2eb]) successfully deleted
I20240502 07:59:09.095675 29651 raft_consensus.cc:2229] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:09.095890 29651 raft_consensus.cc:2258] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:09.098862 22614 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:09.099844 29651 ts_tablet_manager.cc:1875] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:09.103331 29297 consensus_queue.cc:1035] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:09.112890 29651 ts_tablet_manager.cc:1888] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:09.114801 29651 log.cc:1196] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/32e8fea8c9dd4fd88525f8d2d0339970
I20240502 07:59:09.115217 29651 ts_tablet_manager.cc:1909] T 32e8fea8c9dd4fd88525f8d2d0339970 P 1aecef81733c47b3b468504934357231: Deleting consensus metadata
I20240502 07:59:09.115832 24931 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:09.116391 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet 32e8fea8c9dd4fd88525f8d2d0339970 (table rebalancer_dynamic_table_new_name [id=56c1f4ae38dc43749ae3fd1e2fabe2eb]) successfully deleted
I20240502 07:59:09.118283 29297 consensus_queue.cc:1035] T 3d1511fc27c24820afaa6e41161be8bb P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:09.120277 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:09.135921 22351 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:09.137641 29783 tablet_replica.cc:330] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:09.137912 29783 raft_consensus.cc:2229] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:09.138108 29783 raft_consensus.cc:2258] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:09.138569 22743 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:09.138882 29783 ts_tablet_manager.cc:1875] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:09.141371 29783 ts_tablet_manager.cc:1888] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:09.142936 29783 log.cc:1196] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/32e8fea8c9dd4fd88525f8d2d0339970
I20240502 07:59:09.143337 29783 ts_tablet_manager.cc:1909] T 32e8fea8c9dd4fd88525f8d2d0339970 P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:59:09.144157 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet 32e8fea8c9dd4fd88525f8d2d0339970 (table rebalancer_dynamic_table_new_name [id=56c1f4ae38dc43749ae3fd1e2fabe2eb]) successfully deleted
I20240502 07:59:09.164098 22227 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_32"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:09.170727 24931 tablet_service.cc:1467] Processing CreateTablet for tablet ab436438191240a7bed8c3a880ad54b1 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=738584416945405a951925650b8bb983]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:09.171031 24931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab436438191240a7bed8c3a880ad54b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.171397 22351 tablet_service.cc:1467] Processing CreateTablet for tablet ab436438191240a7bed8c3a880ad54b1 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=738584416945405a951925650b8bb983]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:09.171646 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab436438191240a7bed8c3a880ad54b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.171887 22352 tablet_service.cc:1467] Processing CreateTablet for tablet d63f00a2d007442781d58fb938bed480 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=738584416945405a951925650b8bb983]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:09.172137 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d63f00a2d007442781d58fb938bed480. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.172282 25193 tablet_service.cc:1467] Processing CreateTablet for tablet de611cbc61224afa826a3a2361a29a3c (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=738584416945405a951925650b8bb983]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:09.172570 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de611cbc61224afa826a3a2361a29a3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.173669 22483 tablet_service.cc:1467] Processing CreateTablet for tablet ab436438191240a7bed8c3a880ad54b1 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=738584416945405a951925650b8bb983]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:09.173759 24930 tablet_service.cc:1467] Processing CreateTablet for tablet de611cbc61224afa826a3a2361a29a3c (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=738584416945405a951925650b8bb983]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:09.173961 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de611cbc61224afa826a3a2361a29a3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.173962 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab436438191240a7bed8c3a880ad54b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.174932 29789 tablet_bootstrap.cc:492] T de611cbc61224afa826a3a2361a29a3c P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:09.175267 22614 tablet_service.cc:1467] Processing CreateTablet for tablet d63f00a2d007442781d58fb938bed480 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=738584416945405a951925650b8bb983]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:09.175550 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d63f00a2d007442781d58fb938bed480. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.175979 29789 tablet_bootstrap.cc:654] T de611cbc61224afa826a3a2361a29a3c P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.176224 24932 tablet_service.cc:1467] Processing CreateTablet for tablet d63f00a2d007442781d58fb938bed480 (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=738584416945405a951925650b8bb983]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:09.176453 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d63f00a2d007442781d58fb938bed480. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.176772 29788 tablet_bootstrap.cc:492] T d63f00a2d007442781d58fb938bed480 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:09.177291 29787 tablet_bootstrap.cc:492] T ab436438191240a7bed8c3a880ad54b1 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:09.177958 29788 tablet_bootstrap.cc:654] T d63f00a2d007442781d58fb938bed480 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.178663 29787 tablet_bootstrap.cc:654] T ab436438191240a7bed8c3a880ad54b1 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.178880 29790 tablet_bootstrap.cc:492] T ab436438191240a7bed8c3a880ad54b1 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:09.180150 25062 tablet_service.cc:1467] Processing CreateTablet for tablet de611cbc61224afa826a3a2361a29a3c (DEFAULT_TABLE table=rebalancer_extra_table_32 [id=738584416945405a951925650b8bb983]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:09.180269 29790 tablet_bootstrap.cc:654] T ab436438191240a7bed8c3a880ad54b1 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.180442 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de611cbc61224afa826a3a2361a29a3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.180903 29791 tablet_bootstrap.cc:492] T d63f00a2d007442781d58fb938bed480 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:09.182766 29791 tablet_bootstrap.cc:654] T d63f00a2d007442781d58fb938bed480 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.183705 29790 tablet_bootstrap.cc:492] T ab436438191240a7bed8c3a880ad54b1 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:09.184301 29790 ts_tablet_manager.cc:1367] T ab436438191240a7bed8c3a880ad54b1 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:09.184988 29790 raft_consensus.cc:348] T ab436438191240a7bed8c3a880ad54b1 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:09.185145 29790 raft_consensus.cc:374] T ab436438191240a7bed8c3a880ad54b1 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.185479 29790 raft_consensus.cc:729] T ab436438191240a7bed8c3a880ad54b1 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.185810 29787 tablet_bootstrap.cc:492] T ab436438191240a7bed8c3a880ad54b1 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:09.185927 29789 tablet_bootstrap.cc:492] T de611cbc61224afa826a3a2361a29a3c P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:09.185904 29790 consensus_queue.cc:260] T ab436438191240a7bed8c3a880ad54b1 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:09.185971 29787 ts_tablet_manager.cc:1367] T ab436438191240a7bed8c3a880ad54b1 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 07:59:09.186049 29789 ts_tablet_manager.cc:1367] T de611cbc61224afa826a3a2361a29a3c P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 07:59:09.186619 29790 ts_tablet_manager.cc:1398] T ab436438191240a7bed8c3a880ad54b1 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:09.187094 29787 raft_consensus.cc:348] T ab436438191240a7bed8c3a880ad54b1 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:09.187273 29787 raft_consensus.cc:374] T ab436438191240a7bed8c3a880ad54b1 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.187351 29787 raft_consensus.cc:729] T ab436438191240a7bed8c3a880ad54b1 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.187490 29787 consensus_queue.cc:260] T ab436438191240a7bed8c3a880ad54b1 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:09.187778 29787 ts_tablet_manager.cc:1398] T ab436438191240a7bed8c3a880ad54b1 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:09.188102 29791 tablet_bootstrap.cc:492] T d63f00a2d007442781d58fb938bed480 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:09.188194 29791 ts_tablet_manager.cc:1367] T d63f00a2d007442781d58fb938bed480 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:09.188146 29789 raft_consensus.cc:348] T de611cbc61224afa826a3a2361a29a3c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:09.188264 29789 raft_consensus.cc:374] T de611cbc61224afa826a3a2361a29a3c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.188294 29789 raft_consensus.cc:729] T de611cbc61224afa826a3a2361a29a3c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.188406 29789 consensus_queue.cc:260] T de611cbc61224afa826a3a2361a29a3c P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:09.188599 29788 tablet_bootstrap.cc:492] T d63f00a2d007442781d58fb938bed480 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:09.188753 29788 ts_tablet_manager.cc:1367] T d63f00a2d007442781d58fb938bed480 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 07:59:09.189237 29789 ts_tablet_manager.cc:1398] T de611cbc61224afa826a3a2361a29a3c P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:09.189308 29788 raft_consensus.cc:348] T d63f00a2d007442781d58fb938bed480 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:09.189486 29788 raft_consensus.cc:374] T d63f00a2d007442781d58fb938bed480 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.189535 29788 raft_consensus.cc:729] T d63f00a2d007442781d58fb938bed480 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.189687 29788 consensus_queue.cc:260] T d63f00a2d007442781d58fb938bed480 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:09.189785 22227 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:09.190100 29788 ts_tablet_manager.cc:1398] T d63f00a2d007442781d58fb938bed480 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:09.190148 29787 tablet_bootstrap.cc:492] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:09.190275 29788 tablet_bootstrap.cc:492] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:09.191267 29788 tablet_bootstrap.cc:654] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.191404 29787 tablet_bootstrap.cc:654] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.193265 29791 raft_consensus.cc:348] T d63f00a2d007442781d58fb938bed480 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:09.193496 29791 raft_consensus.cc:374] T d63f00a2d007442781d58fb938bed480 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.193588 29791 raft_consensus.cc:729] T d63f00a2d007442781d58fb938bed480 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.194818 29787 tablet_bootstrap.cc:492] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:09.194948 29787 ts_tablet_manager.cc:1367] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:09.195428 29787 raft_consensus.cc:348] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:09.195566 29787 raft_consensus.cc:374] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.195636 29787 raft_consensus.cc:729] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.195771 29787 consensus_queue.cc:260] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:09.196113 29791 consensus_queue.cc:260] T d63f00a2d007442781d58fb938bed480 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:09.196329 25193 tablet_service.cc:1467] Processing CreateTablet for tablet 5b1e3ef842cd43fdab7b9a255cdd3c7b (DEFAULT_TABLE table=rebalancer_dynamic_table [id=7394caf9f3f84c9fbd939cafe5112b7f]), partition=
I20240502 07:59:09.196429 29791 ts_tablet_manager.cc:1398] T d63f00a2d007442781d58fb938bed480 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.008s user 0.000s sys 0.001s
I20240502 07:59:09.196507 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 5b1e3ef842cd43fdab7b9a255cdd3c7b (DEFAULT_TABLE table=rebalancer_dynamic_table [id=7394caf9f3f84c9fbd939cafe5112b7f]), partition=
I20240502 07:59:09.196612 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b1e3ef842cd43fdab7b9a255cdd3c7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.196691 29787 ts_tablet_manager.cc:1398] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:09.196732 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b1e3ef842cd43fdab7b9a255cdd3c7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.197161 29796 tablet_bootstrap.cc:492] T de611cbc61224afa826a3a2361a29a3c P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:09.197537 29787 tablet_bootstrap.cc:492] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:09.198122 29796 tablet_bootstrap.cc:654] T de611cbc61224afa826a3a2361a29a3c P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.199126 29787 tablet_bootstrap.cc:654] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.199155 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 5b1e3ef842cd43fdab7b9a255cdd3c7b (DEFAULT_TABLE table=rebalancer_dynamic_table [id=7394caf9f3f84c9fbd939cafe5112b7f]), partition=
I20240502 07:59:09.199396 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b1e3ef842cd43fdab7b9a255cdd3c7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.199556 29788 tablet_bootstrap.cc:492] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:09.199641 29788 ts_tablet_manager.cc:1367] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20240502 07:59:09.201040 29787 tablet_bootstrap.cc:492] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:09.201123 29787 ts_tablet_manager.cc:1367] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:09.201627 29787 raft_consensus.cc:348] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:09.201733 29787 raft_consensus.cc:374] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.201762 29787 raft_consensus.cc:729] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.201890 29787 consensus_queue.cc:260] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:09.202165 29787 ts_tablet_manager.cc:1398] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:09.202328 29787 tablet_bootstrap.cc:492] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:09.202411 29789 tablet_bootstrap.cc:492] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:09.202379 29788 raft_consensus.cc:348] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:09.202498 29788 raft_consensus.cc:374] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.202528 29788 raft_consensus.cc:729] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.203199 29787 tablet_bootstrap.cc:654] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.203392 29789 tablet_bootstrap.cc:654] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.202641 29788 consensus_queue.cc:260] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:09.204280 29806 tablet_bootstrap.cc:492] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:09.204532 29787 tablet_bootstrap.cc:492] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:09.204622 29787 ts_tablet_manager.cc:1367] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:09.205040 29787 raft_consensus.cc:348] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:09.205133 29787 raft_consensus.cc:374] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.205164 29787 raft_consensus.cc:729] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.205389 29788 ts_tablet_manager.cc:1398] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20240502 07:59:09.206187 29796 tablet_bootstrap.cc:492] T de611cbc61224afa826a3a2361a29a3c P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:09.206341 29796 ts_tablet_manager.cc:1367] T de611cbc61224afa826a3a2361a29a3c P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 07:59:09.206604 29806 tablet_bootstrap.cc:654] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.207039 29789 tablet_bootstrap.cc:492] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:09.207126 29789 ts_tablet_manager.cc:1367] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:09.207131 29787 consensus_queue.cc:260] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:09.207409 29787 ts_tablet_manager.cc:1398] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:09.207628 29789 raft_consensus.cc:348] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:09.207727 29789 raft_consensus.cc:374] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.207758 29789 raft_consensus.cc:729] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.207870 29789 consensus_queue.cc:260] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:09.208133 29789 ts_tablet_manager.cc:1398] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:09.207846 29796 raft_consensus.cc:348] T de611cbc61224afa826a3a2361a29a3c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:09.210019 29796 raft_consensus.cc:374] T de611cbc61224afa826a3a2361a29a3c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.210055 29806 tablet_bootstrap.cc:492] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:09.210134 29806 ts_tablet_manager.cc:1367] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:09.210127 29796 raft_consensus.cc:729] T de611cbc61224afa826a3a2361a29a3c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.210335 29796 consensus_queue.cc:260] T de611cbc61224afa826a3a2361a29a3c P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:09.210667 29796 ts_tablet_manager.cc:1398] T de611cbc61224afa826a3a2361a29a3c P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:09.210606 29806 raft_consensus.cc:348] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:09.210709 29806 raft_consensus.cc:374] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.210745 29806 raft_consensus.cc:729] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.210858 29806 consensus_queue.cc:260] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:09.211135 29806 ts_tablet_manager.cc:1398] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:09.238108 25215 consensus_queue.cc:237] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:09.239523 22501 raft_consensus.cc:1261] T af4d275b38ab49e7ac63e2fb9ffd8c07 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:09.239593 24951 raft_consensus.cc:1261] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:09.240000 28593 consensus_queue.cc:1035] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:09.241039 28593 consensus_queue.cc:1035] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:09.243249 29825 raft_consensus.cc:2932] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:09.243494 24951 raft_consensus.cc:2932] T af4d275b38ab49e7ac63e2fb9ffd8c07 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:09.243533 22501 raft_consensus.cc:2932] T af4d275b38ab49e7ac63e2fb9ffd8c07 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:09.246381 22227 catalog_manager.cc:5484] T af4d275b38ab49e7ac63e2fb9ffd8c07 P fabddb16c85944648a710ec52fc7244c reported cstate change: config changed from index 2 to 3, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:09.250829 22743 tablet_service.cc:1514] Processing DeleteTablet for tablet af4d275b38ab49e7ac63e2fb9ffd8c07 with delete_type TABLET_DATA_TOMBSTONED (TS f00f3a1876444ad2bf9c919068ab8014 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:09.251196 29826 tablet_replica.cc:330] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:09.251444 29826 raft_consensus.cc:2229] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:09.251629 29826 raft_consensus.cc:2258] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:09.252115 29826 ts_tablet_manager.cc:1875] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:09.256103 29826 ts_tablet_manager.cc:1888] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:09.256253 29826 log.cc:1196] T af4d275b38ab49e7ac63e2fb9ffd8c07 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/af4d275b38ab49e7ac63e2fb9ffd8c07
I20240502 07:59:09.256943 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet af4d275b38ab49e7ac63e2fb9ffd8c07 (table rebalancer_extra_table_4 [id=7fe3829782fb4a98912517ae49651ba5]) successfully deleted
I20240502 07:59:09.257694 29729 raft_consensus.cc:482] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:09.257782 29729 raft_consensus.cc:504] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:09.258132 29729 leader_election.cc:288] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:09.258620 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d63f00a2d007442781d58fb938bed480" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:09.258754 22372 raft_consensus.cc:2454] T d63f00a2d007442781d58fb938bed480 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:09.258780 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d63f00a2d007442781d58fb938bed480" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:09.258911 22632 raft_consensus.cc:2454] T d63f00a2d007442781d58fb938bed480 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:09.259215 24888 leader_election.cc:302] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:09.259642 29729 raft_consensus.cc:2781] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:09.259706 29729 raft_consensus.cc:482] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:09.259753 29729 raft_consensus.cc:3037] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:09.260726 29729 raft_consensus.cc:504] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:09.261121 29729 leader_election.cc:288] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:09.261503 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d63f00a2d007442781d58fb938bed480" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:09.261631 22372 raft_consensus.cc:3037] T d63f00a2d007442781d58fb938bed480 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:09.262171 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d63f00a2d007442781d58fb938bed480" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:09.262311 22632 raft_consensus.cc:3037] T d63f00a2d007442781d58fb938bed480 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:09.262562 22372 raft_consensus.cc:2454] T d63f00a2d007442781d58fb938bed480 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:09.262861 24888 leader_election.cc:302] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:09.263069 29729 raft_consensus.cc:2781] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:09.263163 29729 raft_consensus.cc:686] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:09.263304 22632 raft_consensus.cc:2454] T d63f00a2d007442781d58fb938bed480 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:09.263309 29729 consensus_queue.cc:237] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:09.265666 22225 catalog_manager.cc:5484] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:09.297814 29825 raft_consensus.cc:482] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:09.297930 29825 raft_consensus.cc:504] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:09.298292 29825 leader_election.cc:288] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:09.298633 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b1e3ef842cd43fdab7b9a255cdd3c7b" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:09.298633 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b1e3ef842cd43fdab7b9a255cdd3c7b" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:09.298763 24951 raft_consensus.cc:2454] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:09.298763 24821 raft_consensus.cc:2454] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:09.299046 25151 leader_election.cc:302] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:09.299188 29825 raft_consensus.cc:2781] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:09.299248 29825 raft_consensus.cc:482] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:09.299297 29825 raft_consensus.cc:3037] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:09.300056 29825 raft_consensus.cc:504] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:09.300350 29825 leader_election.cc:288] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:09.300642 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b1e3ef842cd43fdab7b9a255cdd3c7b" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:09.300665 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b1e3ef842cd43fdab7b9a255cdd3c7b" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:09.300751 24821 raft_consensus.cc:3037] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:09.300776 24951 raft_consensus.cc:3037] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:09.301699 24821 raft_consensus.cc:2454] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:09.301800 24951 raft_consensus.cc:2454] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:09.302064 25151 leader_election.cc:302] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:09.302201 29825 raft_consensus.cc:2781] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:09.302282 29825 raft_consensus.cc:686] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:09.302410 29825 consensus_queue.cc:237] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:09.304534 22225 catalog_manager.cc:5484] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:09.318339 29729 raft_consensus.cc:482] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:09.318457 29729 raft_consensus.cc:504] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:09.318851 29729 leader_election.cc:288] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:09.319141 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de611cbc61224afa826a3a2361a29a3c" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:09.319279 25215 raft_consensus.cc:2454] T de611cbc61224afa826a3a2361a29a3c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:09.319257 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de611cbc61224afa826a3a2361a29a3c" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:09.319384 25084 raft_consensus.cc:2454] T de611cbc61224afa826a3a2361a29a3c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:09.319757 24885 leader_election.cc:302] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
I20240502 07:59:09.319764 29808 raft_consensus.cc:482] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:09.319859 29808 raft_consensus.cc:504] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:09.319911 29729 raft_consensus.cc:2781] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:09.319952 29729 raft_consensus.cc:482] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:09.319988 29729 raft_consensus.cc:3037] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:09.320236 29808 leader_election.cc:288] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:09.320595 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab436438191240a7bed8c3a880ad54b1" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:09.320708 22501 raft_consensus.cc:2454] T ab436438191240a7bed8c3a880ad54b1 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:09.320941 22305 leader_election.cc:302] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:09.321079 29808 raft_consensus.cc:2781] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:09.321125 29808 raft_consensus.cc:482] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:09.321151 29808 raft_consensus.cc:3037] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:09.321668 29729 raft_consensus.cc:504] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:09.321934 29729 leader_election.cc:288] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:09.322326 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de611cbc61224afa826a3a2361a29a3c" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:09.322348 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab436438191240a7bed8c3a880ad54b1" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:09.322351 29808 raft_consensus.cc:504] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:09.322443 25084 raft_consensus.cc:3037] T de611cbc61224afa826a3a2361a29a3c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:09.322439 24951 raft_consensus.cc:2454] T ab436438191240a7bed8c3a880ad54b1 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:09.322698 29808 leader_election.cc:288] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:09.322853 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de611cbc61224afa826a3a2361a29a3c" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:09.322957 25215 raft_consensus.cc:3037] T de611cbc61224afa826a3a2361a29a3c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:09.323163 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab436438191240a7bed8c3a880ad54b1" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:09.323266 22501 raft_consensus.cc:3037] T ab436438191240a7bed8c3a880ad54b1 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:09.323431 25084 raft_consensus.cc:2454] T de611cbc61224afa826a3a2361a29a3c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:09.323715 24885 leader_election.cc:302] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
I20240502 07:59:09.323951 25215 raft_consensus.cc:2454] T de611cbc61224afa826a3a2361a29a3c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:09.323953 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab436438191240a7bed8c3a880ad54b1" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:09.324039 24951 raft_consensus.cc:3037] T ab436438191240a7bed8c3a880ad54b1 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:09.324213 22501 raft_consensus.cc:2454] T ab436438191240a7bed8c3a880ad54b1 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:09.324846 22305 leader_election.cc:302] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:09.324906 24951 raft_consensus.cc:2454] T ab436438191240a7bed8c3a880ad54b1 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:09.324986 29808 raft_consensus.cc:2781] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:09.325050 29808 raft_consensus.cc:686] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:59:09.325155 29808 consensus_queue.cc:237] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:09.325661 29729 raft_consensus.cc:2781] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:09.325747 29729 raft_consensus.cc:686] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:09.325847 29729 consensus_queue.cc:237] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:09.327347 22225 catalog_manager.cc:5484] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:09.327916 22224 catalog_manager.cc:5484] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:09.383579 24800 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:09.390053 24932 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:09.390357 25193 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:09.405424 22743 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:09.408013 22228 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:09.409003 22351 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:09.418138 24951 raft_consensus.cc:1261] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:09.418737 29609 consensus_queue.cc:1035] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:09.418953 24821 raft_consensus.cc:1261] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:09.419389 29609 consensus_queue.cc:1035] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:09.419760 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:09.422452 22223 catalog_manager.cc:5576] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661) does not have the latest schema for tablet 5b1e3ef842cd43fdab7b9a255cdd3c7b (table rebalancer_dynamic_table [id=7394caf9f3f84c9fbd939cafe5112b7f]). Expected version 1 got 0
I20240502 07:59:09.423347 25062 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:09.424463 22223 catalog_manager.cc:5576] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995) does not have the latest schema for tablet 5b1e3ef842cd43fdab7b9a255cdd3c7b (table rebalancer_dynamic_table [id=7394caf9f3f84c9fbd939cafe5112b7f]). Expected version 1 got 0
I20240502 07:59:09.425448 22614 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:09.425808 29655 tablet.cc:1722] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:09.426162 29655 tablet.cc:1620] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:09.426501 29650 tablet.cc:1722] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:09.426791 29650 tablet.cc:1620] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:09.428053 22223 catalog_manager.cc:5576] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995) does not have the latest schema for tablet 5b1e3ef842cd43fdab7b9a255cdd3c7b (table rebalancer_dynamic_table [id=7394caf9f3f84c9fbd939cafe5112b7f]). Expected version 1 got 0
I20240502 07:59:09.433508 22223 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=7394caf9f3f84c9fbd939cafe5112b7f] alter complete (version 1)
I20240502 07:59:09.448355 29706 mvcc.cc:205] Tried to move back new op lower bound from 7023152125600985088 to 7023152125143904256. Current Snapshot: MvccSnapshot[applied={T|T < 7023152125600985088}]
I20240502 07:59:09.450446 22228 catalog_manager.cc:5576] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661) does not have the latest schema for tablet 5b1e3ef842cd43fdab7b9a255cdd3c7b (table rebalancer_dynamic_table [id=7394caf9f3f84c9fbd939cafe5112b7f]). Expected version 1 got 0
I20240502 07:59:09.478353 29910 tablet.cc:1722] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:09.482766 29910 tablet.cc:1620] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:09.518291 24951 consensus_queue.cc:237] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:09.519886 22764 raft_consensus.cc:1261] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 07:59:09.520412 29729 consensus_queue.cc:1035] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:09.520953 22500 raft_consensus.cc:1261] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c [term 2 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 07:59:09.521596 29729 consensus_queue.cc:1035] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:09.523276 28686 raft_consensus.cc:2932] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:09.523728 22764 raft_consensus.cc:2932] T c5841aadb46447e685af47b70d0feff4 P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:09.523882 22500 raft_consensus.cc:2932] T c5841aadb46447e685af47b70d0feff4 P fabddb16c85944648a710ec52fc7244c [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:09.526871 22223 catalog_manager.cc:5484] T c5841aadb46447e685af47b70d0feff4 P 1aecef81733c47b3b468504934357231 reported cstate change: config changed from index 4 to 6, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New cstate: current_term: 2 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:09.530547 22614 tablet_service.cc:1514] Processing DeleteTablet for tablet c5841aadb46447e685af47b70d0feff4 with delete_type TABLET_DATA_TOMBSTONED (TS e51b10af49484aaa8f0267aebc1bf27f not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:09.530920 29930 tablet_replica.cc:330] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:09.531138 29930 raft_consensus.cc:2229] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:09.531307 29930 raft_consensus.cc:2258] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:09.532613 29930 ts_tablet_manager.cc:1875] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:09.534719 29930 ts_tablet_manager.cc:1888] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 07:59:09.534866 29930 log.cc:1196] T c5841aadb46447e685af47b70d0feff4 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/c5841aadb46447e685af47b70d0feff4
I20240502 07:59:09.535362 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet c5841aadb46447e685af47b70d0feff4 (table rebalance_test_table_1 [id=be3f9b3a6e5147b0afe27972b50219ce]) successfully deleted
I20240502 07:59:09.556586 22224 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:09.561967 29655 tablet.cc:1722] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:09.562325 29655 tablet.cc:1620] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:09.562342 29650 tablet.cc:1722] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:09.562611 29650 tablet.cc:1620] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:09.564764 29910 tablet.cc:1722] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:09.565059 29910 tablet.cc:1620] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:09.569496 22228 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=7394caf9f3f84c9fbd939cafe5112b7f] alter complete (version 2)
I20240502 07:59:09.649857 25193 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:09.650588 25062 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:09.659304 24932 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:09.661495 24800 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:09.662521 22614 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:09.669472 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:09.670145 22351 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:09.670997 22743 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:09.673105 22223 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:09.683758 29655 tablet.cc:1722] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:09.684664 29910 tablet.cc:1722] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:09.686020 29650 tablet.cc:1722] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:09.689568 22221 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=7394caf9f3f84c9fbd939cafe5112b7f] alter complete (version 3)
I20240502 07:59:09.697057 29929 consensus_queue.cc:1035] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:09.702767 29808 consensus_queue.cc:1035] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:09.706514 29929 consensus_queue.cc:1035] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:09.719089 29459 consensus_queue.cc:1035] T ab436438191240a7bed8c3a880ad54b1 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:09.725708 28686 consensus_queue.cc:1035] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:09.729233 29729 consensus_queue.cc:1035] T de611cbc61224afa826a3a2361a29a3c P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:09.751286 24951 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c651077242584f57a165073de3e4a088"
dest_uuid: "1aecef81733c47b3b468504934357231"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:43860
I20240502 07:59:09.751456 24951 raft_consensus.cc:595] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:09.792120 22228 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:09.792320 22228 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:09.794744 22228 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 5b1e3ef842cd43fdab7b9a255cdd3c7b
I20240502 07:59:09.795575 24800 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b1e3ef842cd43fdab7b9a255cdd3c7b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:09 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:59:09.795786 25193 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b1e3ef842cd43fdab7b9a255cdd3c7b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:09 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:09.796403 24932 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b1e3ef842cd43fdab7b9a255cdd3c7b with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:09 UTC) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:09.796576 30035 tablet_replica.cc:330] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:09.796723 30035 raft_consensus.cc:2229] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:09.796931 30035 raft_consensus.cc:2258] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:09.797540 30035 ts_tablet_manager.cc:1875] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:09.798032 30036 tablet_replica.cc:330] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:09.798216 30036 raft_consensus.cc:2229] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:09.798337 30036 raft_consensus.cc:2258] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:09.798574 30034 tablet_replica.cc:330] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:59:09.798756 30034 raft_consensus.cc:2229] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:09.798913 30034 raft_consensus.cc:2258] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:09.798964 30036 ts_tablet_manager.cc:1875] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:09.800099 30035 ts_tablet_manager.cc:1888] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:09.800222 30035 log.cc:1196] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/5b1e3ef842cd43fdab7b9a255cdd3c7b
I20240502 07:59:09.800352 30034 ts_tablet_manager.cc:1875] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:09.800626 30035 ts_tablet_manager.cc:1909] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:59:09.801657 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 5b1e3ef842cd43fdab7b9a255cdd3c7b (table rebalancer_dynamic_table_new_name [id=7394caf9f3f84c9fbd939cafe5112b7f]) successfully deleted
I20240502 07:59:09.802690 30036 ts_tablet_manager.cc:1888] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:09.802825 30036 log.cc:1196] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/5b1e3ef842cd43fdab7b9a255cdd3c7b
I20240502 07:59:09.802978 30034 ts_tablet_manager.cc:1888] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:09.803084 30034 log.cc:1196] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/5b1e3ef842cd43fdab7b9a255cdd3c7b
I20240502 07:59:09.803186 30036 ts_tablet_manager.cc:1909] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 1aecef81733c47b3b468504934357231: Deleting consensus metadata
I20240502 07:59:09.803467 30034 ts_tablet_manager.cc:1909] T 5b1e3ef842cd43fdab7b9a255cdd3c7b P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:59:09.803963 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet 5b1e3ef842cd43fdab7b9a255cdd3c7b (table rebalancer_dynamic_table_new_name [id=7394caf9f3f84c9fbd939cafe5112b7f]) successfully deleted
I20240502 07:59:09.804615 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet 5b1e3ef842cd43fdab7b9a255cdd3c7b (table rebalancer_dynamic_table_new_name [id=7394caf9f3f84c9fbd939cafe5112b7f]) successfully deleted
I20240502 07:59:09.843036 22225 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_34"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:09.851123 25193 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:09.851230 22614 tablet_service.cc:1467] Processing CreateTablet for tablet e9e7a4590c494297bc978a8936fce17f (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=4ca845eadd134f669870771d5a922e65]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:09.851310 25062 tablet_service.cc:1467] Processing CreateTablet for tablet e9e7a4590c494297bc978a8936fce17f (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=4ca845eadd134f669870771d5a922e65]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:09.851552 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9e7a4590c494297bc978a8936fce17f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.851549 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9e7a4590c494297bc978a8936fce17f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.851790 24932 tablet_service.cc:1467] Processing CreateTablet for tablet e9e7a4590c494297bc978a8936fce17f (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=4ca845eadd134f669870771d5a922e65]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:09.852041 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9e7a4590c494297bc978a8936fce17f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.851999 22613 tablet_service.cc:1467] Processing CreateTablet for tablet f1b03db79c944eefb303c60430f46a22 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=4ca845eadd134f669870771d5a922e65]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:09.852205 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1b03db79c944eefb303c60430f46a22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.853734 25193 tablet_service.cc:1467] Processing CreateTablet for tablet f1b03db79c944eefb303c60430f46a22 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=4ca845eadd134f669870771d5a922e65]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:09.853986 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1b03db79c944eefb303c60430f46a22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.854122 22483 tablet_service.cc:1467] Processing CreateTablet for tablet e8534d84d3ce44b6aa74767f2da695cf (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=4ca845eadd134f669870771d5a922e65]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:09.854380 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8534d84d3ce44b6aa74767f2da695cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.855710 24932 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:09.856473 24932 tablet_service.cc:1467] Processing CreateTablet for tablet e8534d84d3ce44b6aa74767f2da695cf (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=4ca845eadd134f669870771d5a922e65]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:09.856725 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8534d84d3ce44b6aa74767f2da695cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.857038 24800 tablet_service.cc:1467] Processing CreateTablet for tablet e8534d84d3ce44b6aa74767f2da695cf (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=4ca845eadd134f669870771d5a922e65]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:09.857293 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8534d84d3ce44b6aa74767f2da695cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.858502 24799 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:09.858855 24930 tablet_service.cc:1467] Processing CreateTablet for tablet f1b03db79c944eefb303c60430f46a22 (DEFAULT_TABLE table=rebalancer_extra_table_34 [id=4ca845eadd134f669870771d5a922e65]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:09.859076 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1b03db79c944eefb303c60430f46a22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.861105 30051 tablet_bootstrap.cc:492] T f1b03db79c944eefb303c60430f46a22 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:09.862241 30051 tablet_bootstrap.cc:654] T f1b03db79c944eefb303c60430f46a22 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.862655 30050 tablet_bootstrap.cc:492] T e9e7a4590c494297bc978a8936fce17f P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:09.863603 30050 tablet_bootstrap.cc:654] T e9e7a4590c494297bc978a8936fce17f P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.865921 30049 tablet_bootstrap.cc:492] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:09.866101 30054 tablet_bootstrap.cc:492] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:09.867905 30054 tablet_bootstrap.cc:654] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.868834 30049 tablet_bootstrap.cc:654] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.870556 22483 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:09.870857 25062 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:09.872121 30054 tablet_bootstrap.cc:492] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:09.872272 30054 ts_tablet_manager.cc:1367] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:09.872462 30051 tablet_bootstrap.cc:492] T f1b03db79c944eefb303c60430f46a22 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:09.872592 30051 ts_tablet_manager.cc:1367] T f1b03db79c944eefb303c60430f46a22 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 07:59:09.873195 30051 raft_consensus.cc:348] T f1b03db79c944eefb303c60430f46a22 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:09.873360 30051 raft_consensus.cc:374] T f1b03db79c944eefb303c60430f46a22 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.873414 30051 raft_consensus.cc:729] T f1b03db79c944eefb303c60430f46a22 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.874380 30051 consensus_queue.cc:260] T f1b03db79c944eefb303c60430f46a22 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:09.874627 30054 raft_consensus.cc:348] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:09.874728 30051 ts_tablet_manager.cc:1398] T f1b03db79c944eefb303c60430f46a22 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:09.874786 30054 raft_consensus.cc:374] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.874890 30054 raft_consensus.cc:729] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.875036 30054 consensus_queue.cc:260] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:09.875448 30054 ts_tablet_manager.cc:1398] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:09.876461 22743 tablet_service.cc:1429] Tablet server has 6 leaders and 0 scanners
I20240502 07:59:09.881263 29297 raft_consensus.cc:482] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:09.881372 29297 raft_consensus.cc:504] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:09.881744 29297 leader_election.cc:288] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:09.882207 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8534d84d3ce44b6aa74767f2da695cf" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
W20240502 07:59:09.882604 24757 leader_election.cc:341] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): Illegal state: must be running to vote when last-logged opid is not known
I20240502 07:59:09.886142 22500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8534d84d3ce44b6aa74767f2da695cf" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
W20240502 07:59:09.886631 24754 leader_election.cc:341] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): Illegal state: must be running to vote when last-logged opid is not known
I20240502 07:59:09.886703 24754 leader_election.cc:302] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b; no voters: 1aecef81733c47b3b468504934357231, fabddb16c85944648a710ec52fc7244c
I20240502 07:59:09.886873 29297 raft_consensus.cc:2726] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 07:59:09.895488 22613 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:09.896631 22230 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:09.902539 30052 tablet_bootstrap.cc:492] T e9e7a4590c494297bc978a8936fce17f P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:09.902607 22351 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:09.903821 30052 tablet_bootstrap.cc:654] T e9e7a4590c494297bc978a8936fce17f P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.912802 30053 tablet_bootstrap.cc:492] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:09.913610 25193 tablet_service.cc:1467] Processing CreateTablet for tablet 218d8ccbf79345bd888cb589cba8aa9d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=05c0c99746a442f99f31bd008825050c]), partition=
I20240502 07:59:09.913792 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 218d8ccbf79345bd888cb589cba8aa9d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=05c0c99746a442f99f31bd008825050c]), partition=
I20240502 07:59:09.913873 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 218d8ccbf79345bd888cb589cba8aa9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.914023 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 218d8ccbf79345bd888cb589cba8aa9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.913993 25062 tablet_service.cc:1467] Processing CreateTablet for tablet 218d8ccbf79345bd888cb589cba8aa9d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=05c0c99746a442f99f31bd008825050c]), partition=
I20240502 07:59:09.914237 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 218d8ccbf79345bd888cb589cba8aa9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:09.914455 30053 tablet_bootstrap.cc:654] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.916070 30051 tablet_bootstrap.cc:492] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:09.917048 30051 tablet_bootstrap.cc:654] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.927618 30061 tablet_bootstrap.cc:492] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:09.929224 30061 tablet_bootstrap.cc:654] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.933583 30052 tablet_bootstrap.cc:492] T e9e7a4590c494297bc978a8936fce17f P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:09.933696 30052 ts_tablet_manager.cc:1367] T e9e7a4590c494297bc978a8936fce17f P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.070s user 0.002s sys 0.000s
I20240502 07:59:09.934167 30052 raft_consensus.cc:348] T e9e7a4590c494297bc978a8936fce17f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:09.934275 30052 raft_consensus.cc:374] T e9e7a4590c494297bc978a8936fce17f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.934302 30052 raft_consensus.cc:729] T e9e7a4590c494297bc978a8936fce17f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.934420 30052 consensus_queue.cc:260] T e9e7a4590c494297bc978a8936fce17f P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:09.934701 30052 ts_tablet_manager.cc:1398] T e9e7a4590c494297bc978a8936fce17f P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:09.934862 30052 tablet_bootstrap.cc:492] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:09.935819 30052 tablet_bootstrap.cc:654] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.942028 30052 tablet_bootstrap.cc:492] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:09.942119 30052 ts_tablet_manager.cc:1367] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:09.942574 30052 raft_consensus.cc:348] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:09.942682 30052 raft_consensus.cc:374] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.942708 30052 raft_consensus.cc:729] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.942814 30052 consensus_queue.cc:260] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:09.944546 30052 ts_tablet_manager.cc:1398] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:09.950431 30061 tablet_bootstrap.cc:492] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:09.950539 30061 ts_tablet_manager.cc:1367] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20240502 07:59:09.951300 30061 raft_consensus.cc:348] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:09.951421 30061 raft_consensus.cc:374] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.951452 30061 raft_consensus.cc:729] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.951598 30061 consensus_queue.cc:260] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:09.951967 30061 ts_tablet_manager.cc:1398] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:09.952677 30051 tablet_bootstrap.cc:492] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:09.952782 30051 ts_tablet_manager.cc:1367] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.000s
I20240502 07:59:09.952908 30053 tablet_bootstrap.cc:492] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:09.953011 30053 ts_tablet_manager.cc:1367] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.000s
I20240502 07:59:09.953361 30050 tablet_bootstrap.cc:492] T e9e7a4590c494297bc978a8936fce17f P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:09.953287 30051 raft_consensus.cc:348] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:09.953433 30051 raft_consensus.cc:374] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.953450 30050 ts_tablet_manager.cc:1367] T e9e7a4590c494297bc978a8936fce17f P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.091s user 0.002s sys 0.000s
I20240502 07:59:09.953478 30051 raft_consensus.cc:729] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.953537 30049 tablet_bootstrap.cc:492] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:09.953536 30053 raft_consensus.cc:348] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:09.953624 30049 ts_tablet_manager.cc:1367] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.088s user 0.001s sys 0.002s
I20240502 07:59:09.953606 30051 consensus_queue.cc:260] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:09.953644 30053 raft_consensus.cc:374] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.953676 30053 raft_consensus.cc:729] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.953799 30053 consensus_queue.cc:260] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:09.953861 30051 ts_tablet_manager.cc:1398] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:09.953904 30050 raft_consensus.cc:348] T e9e7a4590c494297bc978a8936fce17f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:09.954002 30050 raft_consensus.cc:374] T e9e7a4590c494297bc978a8936fce17f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.954032 30050 raft_consensus.cc:729] T e9e7a4590c494297bc978a8936fce17f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.954053 30053 ts_tablet_manager.cc:1398] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:09.954077 30049 raft_consensus.cc:348] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:09.954140 30050 consensus_queue.cc:260] T e9e7a4590c494297bc978a8936fce17f P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:09.954183 30049 raft_consensus.cc:374] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.954231 30049 raft_consensus.cc:729] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.954342 30049 consensus_queue.cc:260] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:09.954406 30050 ts_tablet_manager.cc:1398] T e9e7a4590c494297bc978a8936fce17f P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:09.954562 30050 tablet_bootstrap.cc:492] T e8534d84d3ce44b6aa74767f2da695cf P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:09.954603 30049 ts_tablet_manager.cc:1398] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:09.954757 30049 tablet_bootstrap.cc:492] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:09.955425 30050 tablet_bootstrap.cc:654] T e8534d84d3ce44b6aa74767f2da695cf P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.956151 30049 tablet_bootstrap.cc:654] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.956339 30050 tablet_bootstrap.cc:492] T e8534d84d3ce44b6aa74767f2da695cf P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:09.956521 30050 ts_tablet_manager.cc:1367] T e8534d84d3ce44b6aa74767f2da695cf P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:09.956913 30050 raft_consensus.cc:348] T e8534d84d3ce44b6aa74767f2da695cf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:09.957008 30050 raft_consensus.cc:374] T e8534d84d3ce44b6aa74767f2da695cf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.957036 30050 raft_consensus.cc:729] T e8534d84d3ce44b6aa74767f2da695cf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.957155 30050 consensus_queue.cc:260] T e8534d84d3ce44b6aa74767f2da695cf P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:09.957520 30050 ts_tablet_manager.cc:1398] T e8534d84d3ce44b6aa74767f2da695cf P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:09.957762 30050 tablet_bootstrap.cc:492] T f1b03db79c944eefb303c60430f46a22 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:09.962075 30050 tablet_bootstrap.cc:654] T f1b03db79c944eefb303c60430f46a22 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:09.963357 30050 tablet_bootstrap.cc:492] T f1b03db79c944eefb303c60430f46a22 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:09.963454 30050 ts_tablet_manager.cc:1367] T f1b03db79c944eefb303c60430f46a22 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:09.963950 30050 raft_consensus.cc:348] T f1b03db79c944eefb303c60430f46a22 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:09.964062 30050 raft_consensus.cc:374] T f1b03db79c944eefb303c60430f46a22 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.964097 30050 raft_consensus.cc:729] T f1b03db79c944eefb303c60430f46a22 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.964232 30050 consensus_queue.cc:260] T f1b03db79c944eefb303c60430f46a22 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:09.964639 30050 ts_tablet_manager.cc:1398] T f1b03db79c944eefb303c60430f46a22 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:09.968204 30049 tablet_bootstrap.cc:492] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:09.968300 30049 ts_tablet_manager.cc:1367] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.001s
I20240502 07:59:09.968771 30049 raft_consensus.cc:348] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:09.968895 30049 raft_consensus.cc:374] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:09.968926 30049 raft_consensus.cc:729] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:09.969046 30049 consensus_queue.cc:260] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:09.969358 30049 ts_tablet_manager.cc:1398] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:09.985890 28773 raft_consensus.cc:482] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:09.986014 28773 raft_consensus.cc:504] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:09.986442 28773 leader_election.cc:288] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:09.986708 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "218d8ccbf79345bd888cb589cba8aa9d" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:09.986824 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "218d8ccbf79345bd888cb589cba8aa9d" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:09.986851 25215 raft_consensus.cc:2454] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:09.986958 25083 raft_consensus.cc:2454] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:09.987171 22701 leader_election.cc:302] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:09.987341 28773 raft_consensus.cc:2781] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:09.987396 28773 raft_consensus.cc:482] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:09.987435 28773 raft_consensus.cc:3037] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:09.988600 28773 raft_consensus.cc:504] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:09.988960 28773 leader_election.cc:288] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:09.989277 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "218d8ccbf79345bd888cb589cba8aa9d" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:09.989359 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "218d8ccbf79345bd888cb589cba8aa9d" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:09.989411 25083 raft_consensus.cc:3037] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:09.989468 25215 raft_consensus.cc:3037] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:09.990301 25083 raft_consensus.cc:2454] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:09.990510 25215 raft_consensus.cc:2454] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:09.990612 22698 leader_election.cc:302] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:09.990782 28773 raft_consensus.cc:2781] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:09.990948 28773 raft_consensus.cc:686] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:59:09.991089 28773 consensus_queue.cc:237] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:09.993186 22225 catalog_manager.cc:5484] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:10.003427 30066 raft_consensus.cc:482] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:10.003551 30066 raft_consensus.cc:504] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:10.003919 30066 leader_election.cc:288] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:10.004278 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9e7a4590c494297bc978a8936fce17f" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:10.004282 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9e7a4590c494297bc978a8936fce17f" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:10.004407 22632 raft_consensus.cc:2454] T e9e7a4590c494297bc978a8936fce17f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:10.004415 24951 raft_consensus.cc:2454] T e9e7a4590c494297bc978a8936fce17f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:10.004686 25020 leader_election.cc:302] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:10.004863 30066 raft_consensus.cc:2781] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:10.004925 30066 raft_consensus.cc:482] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:10.004954 30066 raft_consensus.cc:3037] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:10.005990 30066 raft_consensus.cc:504] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:10.006318 30066 leader_election.cc:288] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:10.006616 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9e7a4590c494297bc978a8936fce17f" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:10.006763 22632 raft_consensus.cc:3037] T e9e7a4590c494297bc978a8936fce17f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:10.007668 22632 raft_consensus.cc:2454] T e9e7a4590c494297bc978a8936fce17f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:10.008029 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9e7a4590c494297bc978a8936fce17f" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:10.008034 25020 leader_election.cc:302] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:10.008147 24951 raft_consensus.cc:3037] T e9e7a4590c494297bc978a8936fce17f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:10.009181 24951 raft_consensus.cc:2454] T e9e7a4590c494297bc978a8936fce17f P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:10.009469 30066 raft_consensus.cc:2781] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:10.009557 30066 raft_consensus.cc:686] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:10.009692 30066 consensus_queue.cc:237] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:10.011643 22225 catalog_manager.cc:5484] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:10.025094 22632 consensus_queue.cc:237] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
W20240502 07:59:10.026763 22567 consensus_peers.cc:485] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): Couldn't send request to peer 227eec4b5f014d349661f6e0b39b9055. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0f577dafc0b4846a1a9e4dc54a5a687. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.026770 22500 raft_consensus.cc:1261] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:10.027175 30113 consensus_queue.cc:1035] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:10.029772 22763 raft_consensus.cc:1261] T a0f577dafc0b4846a1a9e4dc54a5a687 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:10.030220 30113 consensus_queue.cc:1035] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:10.032055 27965 raft_consensus.cc:2932] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:10.032490 22763 raft_consensus.cc:2932] T a0f577dafc0b4846a1a9e4dc54a5a687 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:10.034402 22500 raft_consensus.cc:2932] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:10.035274 22225 catalog_manager.cc:5484] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:10.040091 29729 raft_consensus.cc:979] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231: : Instructing follower f00f3a1876444ad2bf9c919068ab8014 to start an election
I20240502 07:59:10.040191 29729 raft_consensus.cc:1067] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Signalling peer f00f3a1876444ad2bf9c919068ab8014 to start an election
I20240502 07:59:10.040716 22763 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c651077242584f57a165073de3e4a088"
dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
from {username='slave'} at 127.15.129.134:51231
I20240502 07:59:10.040859 22763 raft_consensus.cc:482] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:10.040922 22763 raft_consensus.cc:3037] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:10.041903 22763 raft_consensus.cc:504] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:10.042327 22763 leader_election.cc:288] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 2 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:10.042588 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c651077242584f57a165073de3e4a088" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:10.042716 24951 raft_consensus.cc:3032] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:10.042762 24951 raft_consensus.cc:729] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:10.042719 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c651077242584f57a165073de3e4a088" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:10.042810 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c651077242584f57a165073de3e4a088" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:10.042836 25215 raft_consensus.cc:3037] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:10.042922 25083 raft_consensus.cc:3037] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:10.042919 24951 consensus_queue.cc:260] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:10.043128 24951 raft_consensus.cc:3037] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:10.043841 25083 raft_consensus.cc:2454] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 2.
I20240502 07:59:10.043861 25215 raft_consensus.cc:2454] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 2.
I20240502 07:59:10.044142 24951 raft_consensus.cc:2454] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 2.
I20240502 07:59:10.044279 22701 leader_election.cc:302] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:10.045488 28773 raft_consensus.cc:2781] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:10.045567 28773 raft_consensus.cc:686] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [term 2 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:59:10.045680 28773 consensus_queue.cc:237] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:10.047652 22230 catalog_manager.cc:5484] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 1 to 2, leader changed from 1aecef81733c47b3b468504934357231 (127.15.129.134) to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 2 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:10.075294 22500 consensus_queue.cc:237] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:10.076889 22763 raft_consensus.cc:1261] T f1890c9d5c664106bacc97d440c9cbc2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:10.076944 22632 raft_consensus.cc:1261] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:10.077159 22439 consensus_peers.cc:485] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c -> Peer 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): Couldn't send request to peer 43930dee0cd5498fa8e357a34dd71432. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f1890c9d5c664106bacc97d440c9cbc2. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.077387 30134 consensus_queue.cc:1035] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:10.078552 30134 consensus_queue.cc:1035] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:10.080509 30135 raft_consensus.cc:2932] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:10.080917 22763 raft_consensus.cc:2932] T f1890c9d5c664106bacc97d440c9cbc2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:10.083212 22632 raft_consensus.cc:2932] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:10.083580 22224 catalog_manager.cc:5484] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c reported cstate change: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:10.098598 22227 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:10.108819 22763 consensus_queue.cc:237] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.109675 25215 raft_consensus.cc:1261] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:10.109771 25083 raft_consensus.cc:1261] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:10.110541 30135 raft_consensus.cc:482] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:10.110641 30135 raft_consensus.cc:504] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:10.110992 30135 leader_election.cc:288] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:10.111200 22632 raft_consensus.cc:1261] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:10.111431 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8534d84d3ce44b6aa74767f2da695cf" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:10.111500 30084 consensus_queue.cc:1035] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:10.111564 24952 raft_consensus.cc:2454] T e8534d84d3ce44b6aa74767f2da695cf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:10.111819 22439 leader_election.cc:302] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:10.112007 29728 raft_consensus.cc:2781] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:10.112062 29728 raft_consensus.cc:482] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:10.112109 29728 raft_consensus.cc:3037] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
W20240502 07:59:10.112242 22698 consensus_peers.cc:485] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 -> Peer 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): Couldn't send request to peer 227eec4b5f014d349661f6e0b39b9055. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 895f9a31785d4558acd3b64904a645ca. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.112262 22500 raft_consensus.cc:1261] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:10.112236 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8534d84d3ce44b6aa74767f2da695cf" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:10.112349 24820 raft_consensus.cc:2454] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:10.112850 28773 consensus_queue.cc:1035] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:10.113111 28773 consensus_queue.cc:1035] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:10.113153 29728 raft_consensus.cc:504] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:10.113453 28773 consensus_queue.cc:1035] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:10.113487 29728 leader_election.cc:288] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:10.113966 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8534d84d3ce44b6aa74767f2da695cf" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:10.114063 24820 raft_consensus.cc:3037] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:10.114231 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8534d84d3ce44b6aa74767f2da695cf" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:10.114329 24952 raft_consensus.cc:3037] T e8534d84d3ce44b6aa74767f2da695cf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:10.115097 24820 raft_consensus.cc:2454] T e8534d84d3ce44b6aa74767f2da695cf P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:10.115342 24952 raft_consensus.cc:2454] T e8534d84d3ce44b6aa74767f2da695cf P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:10.116261 22439 leader_election.cc:302] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:10.116515 30134 raft_consensus.cc:2781] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:10.116585 30134 raft_consensus.cc:686] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:59:10.116684 22230 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 218d8ccbf79345bd888cb589cba8aa9d (table rebalancer_dynamic_table [id=05c0c99746a442f99f31bd008825050c]). Expected version 1 got 0
I20240502 07:59:10.116700 30134 consensus_queue.cc:237] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:10.118701 22227 catalog_manager.cc:5576] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733) does not have the latest schema for tablet 218d8ccbf79345bd888cb589cba8aa9d (table rebalancer_dynamic_table [id=05c0c99746a442f99f31bd008825050c]). Expected version 1 got 0
I20240502 07:59:10.119653 22230 catalog_manager.cc:5484] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:10.122265 22499 raft_consensus.cc:2932] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:10.123692 30084 raft_consensus.cc:2932] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:10.125102 22230 catalog_manager.cc:5484] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c reported cstate change: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:10.125943 29655 tablet.cc:1722] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:10.126272 29655 tablet.cc:1620] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:10.126881 22632 raft_consensus.cc:2932] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:10.130863 22227 catalog_manager.cc:5576] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733) does not have the latest schema for tablet 218d8ccbf79345bd888cb589cba8aa9d (table rebalancer_dynamic_table [id=05c0c99746a442f99f31bd008825050c]). Expected version 1 got 0
I20240502 07:59:10.133392 22224 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=05c0c99746a442f99f31bd008825050c] alter complete (version 1)
I20240502 07:59:10.133291 30165 tablet.cc:1722] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:10.134227 30165 tablet.cc:1620] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:59:10.136457 30166 tablet.cc:1722] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:10.136884 30166 tablet.cc:1620] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:10.148743 30113 raft_consensus.cc:482] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:10.148869 30113 raft_consensus.cc:504] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:10.149279 30113 leader_election.cc:288] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:10.149663 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1b03db79c944eefb303c60430f46a22" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:10.149796 24951 raft_consensus.cc:2454] T f1b03db79c944eefb303c60430f46a22 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:10.150012 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1b03db79c944eefb303c60430f46a22" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:10.150070 22570 leader_election.cc:302] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:10.150135 25214 raft_consensus.cc:2454] T f1b03db79c944eefb303c60430f46a22 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:10.150285 30113 raft_consensus.cc:2781] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:10.150342 30113 raft_consensus.cc:482] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:10.150393 30113 raft_consensus.cc:3037] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:10.151486 30113 raft_consensus.cc:504] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:10.151777 30113 leader_election.cc:288] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:10.152566 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1b03db79c944eefb303c60430f46a22" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:10.152690 24951 raft_consensus.cc:3037] T f1b03db79c944eefb303c60430f46a22 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:10.153699 24951 raft_consensus.cc:2454] T f1b03db79c944eefb303c60430f46a22 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:10.153905 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1b03db79c944eefb303c60430f46a22" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:10.153968 22570 leader_election.cc:302] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:10.154016 25214 raft_consensus.cc:3037] T f1b03db79c944eefb303c60430f46a22 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:10.154116 30113 raft_consensus.cc:2781] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:10.154186 30113 raft_consensus.cc:686] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:59:10.154302 30113 consensus_queue.cc:237] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:10.154963 25214 raft_consensus.cc:2454] T f1b03db79c944eefb303c60430f46a22 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:10.156754 22224 catalog_manager.cc:5484] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:10.158500 22632 consensus_queue.cc:237] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
W20240502 07:59:10.160081 22570 consensus_peers.cc:485] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f -> Peer f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): Couldn't send request to peer f00f3a1876444ad2bf9c919068ab8014. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 61690f2a278745aeace757c233e75b25. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.160434 25214 raft_consensus.cc:1261] T 61690f2a278745aeace757c233e75b25 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:10.160460 24951 raft_consensus.cc:1261] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:10.160815 30113 consensus_queue.cc:1035] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:10.160975 27965 consensus_queue.cc:1035] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:10.163066 27965 raft_consensus.cc:2932] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:10.163573 25214 raft_consensus.cc:2932] T 61690f2a278745aeace757c233e75b25 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:10.165645 24951 raft_consensus.cc:2932] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:10.167518 22224 catalog_manager.cc:5484] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:10.188675 25083 consensus_queue.cc:237] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:10.190055 25214 raft_consensus.cc:1261] T 6c958599492f48b7a8e4946640073d5c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:10.190168 24820 raft_consensus.cc:1261] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:10.190176 25020 consensus_peers.cc:485] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 -> Peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): Couldn't send request to peer e51b10af49484aaa8f0267aebc1bf27f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c958599492f48b7a8e4946640073d5c. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.190654 30202 consensus_queue.cc:1035] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:10.190858 28464 consensus_queue.cc:1035] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:10.192859 30202 raft_consensus.cc:2932] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:10.193276 24820 raft_consensus.cc:2932] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:10.196537 22225 catalog_manager.cc:5484] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:10.198101 25214 raft_consensus.cc:2932] T 6c958599492f48b7a8e4946640073d5c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:10.215876 22632 consensus_queue.cc:237] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:10.217381 22499 raft_consensus.cc:1261] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:10.217448 22763 raft_consensus.cc:1261] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:10.217799 30116 consensus_queue.cc:1035] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:10.218024 22570 consensus_peers.cc:485] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): Couldn't send request to peer 43930dee0cd5498fa8e357a34dd71432. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2241a741bf85433b9fea512e9c530416. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.218041 29693 consensus_queue.cc:1035] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:10.220716 30116 raft_consensus.cc:2932] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:10.221155 22763 raft_consensus.cc:2932] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:10.224064 22225 catalog_manager.cc:5484] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:10.224052 22499 raft_consensus.cc:2932] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:10.237995 22225 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:10.243732 30165 tablet.cc:1722] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:10.244067 30165 tablet.cc:1620] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:59:10.244849 30166 tablet.cc:1722] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:10.245169 30166 tablet.cc:1620] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:10.247779 29655 tablet.cc:1722] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:10.248078 29655 tablet.cc:1620] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:10.250797 22225 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=05c0c99746a442f99f31bd008825050c] alter complete (version 2)
I20240502 07:59:10.260519 22632 consensus_queue.cc:237] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:10.262117 22763 raft_consensus.cc:1261] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:10.262290 22569 consensus_peers.cc:485] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8477b961ce384edea078851122e196e9. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:10.262547 22372 raft_consensus.cc:1261] T 8477b961ce384edea078851122e196e9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:10.262833 27965 consensus_queue.cc:1035] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:10.263008 30113 consensus_queue.cc:1035] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:10.265578 27965 raft_consensus.cc:2932] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:10.265935 22763 raft_consensus.cc:2932] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:10.268879 22372 raft_consensus.cc:2932] T 8477b961ce384edea078851122e196e9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:10.269387 22224 catalog_manager.cc:5484] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:10.358026 22227 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:10.360018 25062 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:10.364367 24930 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:10.364979 30165 tablet.cc:1722] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:10.368141 29655 tablet.cc:1722] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:10.370167 22613 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:10.372460 22743 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:10.374187 24800 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:10.377815 30166 tablet.cc:1722] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:10.378410 22227 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=05c0c99746a442f99f31bd008825050c] alter complete (version 3)
I20240502 07:59:10.380326 25193 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:10.403285 30202 consensus_queue.cc:1035] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:10.405297 22351 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:10.408725 22483 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:10.435253 30066 consensus_queue.cc:1035] T e9e7a4590c494297bc978a8936fce17f P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:10.444482 30298 ts_tablet_manager.cc:902] T a0f577dafc0b4846a1a9e4dc54a5a687 P 227eec4b5f014d349661f6e0b39b9055: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:10.445029 30298 tablet_copy_client.cc:308] T a0f577dafc0b4846a1a9e4dc54a5a687 P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:59:10.445469 22653 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet a0f577dafc0b4846a1a9e4dc54a5a687 from peer 227eec4b5f014d349661f6e0b39b9055 ({username='slave'} at 127.15.129.135:39233)
I20240502 07:59:10.445600 22653 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet a0f577dafc0b4846a1a9e4dc54a5a687 from peer 227eec4b5f014d349661f6e0b39b9055 at {username='slave'} at 127.15.129.135:39233: session id = 227eec4b5f014d349661f6e0b39b9055-a0f577dafc0b4846a1a9e4dc54a5a687
I20240502 07:59:10.446550 22653 tablet_copy_source_session.cc:205] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.447523 30298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0f577dafc0b4846a1a9e4dc54a5a687. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.449703 30298 tablet_copy_client.cc:783] T a0f577dafc0b4846a1a9e4dc54a5a687 P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.449870 30298 tablet_copy_client.cc:647] T a0f577dafc0b4846a1a9e4dc54a5a687 P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.452535 30298 tablet_copy_client.cc:523] T a0f577dafc0b4846a1a9e4dc54a5a687 P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:10.453939 30298 tablet_bootstrap.cc:492] T a0f577dafc0b4846a1a9e4dc54a5a687 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:10.481871 25084 raft_consensus.cc:1261] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 07:59:10.482378 30156 consensus_queue.cc:1035] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:10.482932 30298 tablet_bootstrap.cc:492] T a0f577dafc0b4846a1a9e4dc54a5a687 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:10.483291 30298 tablet_bootstrap.cc:492] T a0f577dafc0b4846a1a9e4dc54a5a687 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap complete.
I20240502 07:59:10.483505 30298 ts_tablet_manager.cc:1367] T a0f577dafc0b4846a1a9e4dc54a5a687 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.030s user 0.023s sys 0.004s
I20240502 07:59:10.483959 30298 raft_consensus.cc:348] T a0f577dafc0b4846a1a9e4dc54a5a687 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.484113 30298 raft_consensus.cc:729] T a0f577dafc0b4846a1a9e4dc54a5a687 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.484236 30298 consensus_queue.cc:260] T a0f577dafc0b4846a1a9e4dc54a5a687 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.484511 30298 ts_tablet_manager.cc:1398] T a0f577dafc0b4846a1a9e4dc54a5a687 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:10.485049 22653 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session 227eec4b5f014d349661f6e0b39b9055-a0f577dafc0b4846a1a9e4dc54a5a687 received from {username='slave'} at 127.15.129.135:39233
I20240502 07:59:10.485152 22653 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session 227eec4b5f014d349661f6e0b39b9055-a0f577dafc0b4846a1a9e4dc54a5a687 on tablet a0f577dafc0b4846a1a9e4dc54a5a687 with peer 227eec4b5f014d349661f6e0b39b9055
I20240502 07:59:10.485739 25214 raft_consensus.cc:1261] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432 [term 2 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 07:59:10.486974 30155 consensus_queue.cc:1035] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:10.490347 24951 raft_consensus.cc:1261] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 2 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 07:59:10.490752 30164 consensus_queue.cc:1035] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:10.494822 22224 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:10.494961 22224 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:10.497478 22224 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 218d8ccbf79345bd888cb589cba8aa9d
I20240502 07:59:10.498034 25062 tablet_service.cc:1514] Processing DeleteTablet for tablet 218d8ccbf79345bd888cb589cba8aa9d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:10 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:10.498078 22743 tablet_service.cc:1514] Processing DeleteTablet for tablet 218d8ccbf79345bd888cb589cba8aa9d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:10 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:10.498479 30323 tablet_replica.cc:330] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:10.498593 25193 tablet_service.cc:1514] Processing DeleteTablet for tablet 218d8ccbf79345bd888cb589cba8aa9d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:10 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:10.498879 30326 tablet_replica.cc:330] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:10.498903 30323 raft_consensus.cc:2229] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:10.499032 30326 raft_consensus.cc:2229] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:10.499045 30323 raft_consensus.cc:2258] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:10.499159 30326 raft_consensus.cc:2258] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:10.499680 30323 ts_tablet_manager.cc:1875] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:10.499694 30326 ts_tablet_manager.cc:1875] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:10.502257 30326 ts_tablet_manager.cc:1888] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:10.502333 30326 log.cc:1196] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/218d8ccbf79345bd888cb589cba8aa9d
I20240502 07:59:10.502595 30326 ts_tablet_manager.cc:1909] T 218d8ccbf79345bd888cb589cba8aa9d P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:59:10.502974 30324 tablet_replica.cc:330] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:10.503279 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 218d8ccbf79345bd888cb589cba8aa9d (table rebalancer_dynamic_table_new_name [id=05c0c99746a442f99f31bd008825050c]) successfully deleted
I20240502 07:59:10.503471 30324 raft_consensus.cc:2229] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:10.503243 30323 ts_tablet_manager.cc:1888] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:10.503628 30323 log.cc:1196] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/218d8ccbf79345bd888cb589cba8aa9d
I20240502 07:59:10.503857 30324 raft_consensus.cc:2258] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:10.504205 30323 ts_tablet_manager.cc:1909] T 218d8ccbf79345bd888cb589cba8aa9d P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:59:10.504930 30324 ts_tablet_manager.cc:1875] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:10.505131 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet 218d8ccbf79345bd888cb589cba8aa9d (table rebalancer_dynamic_table_new_name [id=05c0c99746a442f99f31bd008825050c]) successfully deleted
I20240502 07:59:10.507278 30324 ts_tablet_manager.cc:1888] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:10.507365 30324 log.cc:1196] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/218d8ccbf79345bd888cb589cba8aa9d
I20240502 07:59:10.507622 30324 ts_tablet_manager.cc:1909] T 218d8ccbf79345bd888cb589cba8aa9d P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:59:10.508225 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 218d8ccbf79345bd888cb589cba8aa9d (table rebalancer_dynamic_table_new_name [id=05c0c99746a442f99f31bd008825050c]) successfully deleted
I20240502 07:59:10.509908 30298 ts_tablet_manager.cc:902] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055: Initiating tablet copy from peer f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:10.510412 30298 tablet_copy_client.cc:308] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.132:41673
I20240502 07:59:10.511044 22785 tablet_copy_service.cc:140] P f00f3a1876444ad2bf9c919068ab8014: Received BeginTabletCopySession request for tablet 895f9a31785d4558acd3b64904a645ca from peer 227eec4b5f014d349661f6e0b39b9055 ({username='slave'} at 127.15.129.135:39217)
I20240502 07:59:10.511207 22785 tablet_copy_service.cc:161] P f00f3a1876444ad2bf9c919068ab8014: Beginning new tablet copy session on tablet 895f9a31785d4558acd3b64904a645ca from peer 227eec4b5f014d349661f6e0b39b9055 at {username='slave'} at 127.15.129.135:39217: session id = 227eec4b5f014d349661f6e0b39b9055-895f9a31785d4558acd3b64904a645ca
I20240502 07:59:10.512089 22785 tablet_copy_source_session.cc:205] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.512980 30298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 895f9a31785d4558acd3b64904a645ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.515425 30298 tablet_copy_client.cc:783] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.515583 30298 tablet_copy_client.cc:647] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.517082 30298 tablet_copy_client.cc:523] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:10.519371 30298 tablet_bootstrap.cc:492] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:10.546005 30298 tablet_bootstrap.cc:492] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:10.546378 30298 tablet_bootstrap.cc:492] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055: Bootstrap complete.
I20240502 07:59:10.546603 30298 ts_tablet_manager.cc:1367] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.027s user 0.026s sys 0.000s
I20240502 07:59:10.547088 30298 raft_consensus.cc:348] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.547241 30298 raft_consensus.cc:729] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.547451 30298 consensus_queue.cc:260] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.547791 30298 ts_tablet_manager.cc:1398] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:10.548399 22785 tablet_copy_service.cc:342] P f00f3a1876444ad2bf9c919068ab8014: Request end of tablet copy session 227eec4b5f014d349661f6e0b39b9055-895f9a31785d4558acd3b64904a645ca received from {username='slave'} at 127.15.129.135:39217
I20240502 07:59:10.548502 22785 tablet_copy_service.cc:433] P f00f3a1876444ad2bf9c919068ab8014: ending tablet copy session 227eec4b5f014d349661f6e0b39b9055-895f9a31785d4558acd3b64904a645ca on tablet 895f9a31785d4558acd3b64904a645ca with peer 227eec4b5f014d349661f6e0b39b9055
I20240502 07:59:10.562013 30116 consensus_queue.cc:1035] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:10.566124 29693 consensus_queue.cc:1035] T f1b03db79c944eefb303c60430f46a22 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:10.571596 30360 ts_tablet_manager.cc:902] T f1890c9d5c664106bacc97d440c9cbc2 P 43930dee0cd5498fa8e357a34dd71432: Initiating tablet copy from peer fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:10.572170 30360 tablet_copy_client.cc:308] T f1890c9d5c664106bacc97d440c9cbc2 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.130:39625
I20240502 07:59:10.572667 22523 tablet_copy_service.cc:140] P fabddb16c85944648a710ec52fc7244c: Received BeginTabletCopySession request for tablet f1890c9d5c664106bacc97d440c9cbc2 from peer 43930dee0cd5498fa8e357a34dd71432 ({username='slave'} at 127.15.129.136:45143)
I20240502 07:59:10.572793 22523 tablet_copy_service.cc:161] P fabddb16c85944648a710ec52fc7244c: Beginning new tablet copy session on tablet f1890c9d5c664106bacc97d440c9cbc2 from peer 43930dee0cd5498fa8e357a34dd71432 at {username='slave'} at 127.15.129.136:45143: session id = 43930dee0cd5498fa8e357a34dd71432-f1890c9d5c664106bacc97d440c9cbc2
I20240502 07:59:10.573801 22523 tablet_copy_source_session.cc:205] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.576560 30360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1890c9d5c664106bacc97d440c9cbc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.578351 29728 consensus_queue.cc:1035] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:10.580519 30360 tablet_copy_client.cc:783] T f1890c9d5c664106bacc97d440c9cbc2 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.581776 30360 tablet_copy_client.cc:647] T f1890c9d5c664106bacc97d440c9cbc2 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.586040 29728 consensus_queue.cc:1035] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:10.587914 30360 tablet_copy_client.cc:523] T f1890c9d5c664106bacc97d440c9cbc2 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:10.589972 30379 ts_tablet_manager.cc:902] T 61690f2a278745aeace757c233e75b25 P f00f3a1876444ad2bf9c919068ab8014: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:10.591291 30379 tablet_copy_client.cc:308] T 61690f2a278745aeace757c233e75b25 P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:59:10.591710 30360 tablet_bootstrap.cc:492] T f1890c9d5c664106bacc97d440c9cbc2 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:10.592185 22653 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet 61690f2a278745aeace757c233e75b25 from peer f00f3a1876444ad2bf9c919068ab8014 ({username='slave'} at 127.15.129.132:52697)
I20240502 07:59:10.592298 22653 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet 61690f2a278745aeace757c233e75b25 from peer f00f3a1876444ad2bf9c919068ab8014 at {username='slave'} at 127.15.129.132:52697: session id = f00f3a1876444ad2bf9c919068ab8014-61690f2a278745aeace757c233e75b25
I20240502 07:59:10.593178 22653 tablet_copy_source_session.cc:205] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.594313 30379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61690f2a278745aeace757c233e75b25. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.596761 30379 tablet_copy_client.cc:783] T 61690f2a278745aeace757c233e75b25 P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.598774 30379 tablet_copy_client.cc:647] T 61690f2a278745aeace757c233e75b25 P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.598804 24930 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:10.600075 22227 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:10.600812 30379 tablet_copy_client.cc:523] T 61690f2a278745aeace757c233e75b25 P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:10.602874 30379 tablet_bootstrap.cc:492] T 61690f2a278745aeace757c233e75b25 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:10.608711 25062 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:10.609356 30379 tablet_bootstrap.cc:492] T 61690f2a278745aeace757c233e75b25 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:10.609552 25193 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:10.623914 25193 tablet_service.cc:1467] Processing CreateTablet for tablet 40da7efedda844eca55b346d041981f9 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=3af9bd84a0ce41f4a2b33787e975d248]), partition=
I20240502 07:59:10.623885 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 40da7efedda844eca55b346d041981f9 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=3af9bd84a0ce41f4a2b33787e975d248]), partition=
I20240502 07:59:10.624140 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40da7efedda844eca55b346d041981f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.624140 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40da7efedda844eca55b346d041981f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.625768 30379 tablet_bootstrap.cc:492] T 61690f2a278745aeace757c233e75b25 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap complete.
I20240502 07:59:10.626084 30379 ts_tablet_manager.cc:1367] T 61690f2a278745aeace757c233e75b25 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.003s
I20240502 07:59:10.626703 25062 tablet_service.cc:1467] Processing CreateTablet for tablet 40da7efedda844eca55b346d041981f9 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=3af9bd84a0ce41f4a2b33787e975d248]), partition=
I20240502 07:59:10.626950 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40da7efedda844eca55b346d041981f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.626873 30379 raft_consensus.cc:348] T 61690f2a278745aeace757c233e75b25 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:10.626994 30379 raft_consensus.cc:729] T 61690f2a278745aeace757c233e75b25 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.627242 30379 consensus_queue.cc:260] T 61690f2a278745aeace757c233e75b25 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:10.629832 30387 ts_tablet_manager.cc:902] T 6c958599492f48b7a8e4946640073d5c P e51b10af49484aaa8f0267aebc1bf27f: Initiating tablet copy from peer 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:10.630992 30379 ts_tablet_manager.cc:1398] T 61690f2a278745aeace757c233e75b25 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:59:10.631021 30387 tablet_copy_client.cc:308] T 6c958599492f48b7a8e4946640073d5c P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.135:42733
I20240502 07:59:10.631423 25104 tablet_copy_service.cc:140] P 227eec4b5f014d349661f6e0b39b9055: Received BeginTabletCopySession request for tablet 6c958599492f48b7a8e4946640073d5c from peer e51b10af49484aaa8f0267aebc1bf27f ({username='slave'} at 127.15.129.131:57623)
I20240502 07:59:10.631520 25104 tablet_copy_service.cc:161] P 227eec4b5f014d349661f6e0b39b9055: Beginning new tablet copy session on tablet 6c958599492f48b7a8e4946640073d5c from peer e51b10af49484aaa8f0267aebc1bf27f at {username='slave'} at 127.15.129.131:57623: session id = e51b10af49484aaa8f0267aebc1bf27f-6c958599492f48b7a8e4946640073d5c
I20240502 07:59:10.632483 25104 tablet_copy_source_session.cc:205] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.633659 22653 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session f00f3a1876444ad2bf9c919068ab8014-61690f2a278745aeace757c233e75b25 received from {username='slave'} at 127.15.129.132:52697
I20240502 07:59:10.633735 22653 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session f00f3a1876444ad2bf9c919068ab8014-61690f2a278745aeace757c233e75b25 on tablet 61690f2a278745aeace757c233e75b25 with peer f00f3a1876444ad2bf9c919068ab8014
I20240502 07:59:10.634064 24800 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:10.636416 30387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c958599492f48b7a8e4946640073d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.639217 22613 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:10.649389 30389 tablet_bootstrap.cc:492] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:10.649888 22351 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:10.650452 22743 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:10.652261 30389 tablet_bootstrap.cc:654] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:10.654843 30388 tablet_bootstrap.cc:492] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:10.656129 30387 tablet_copy_client.cc:783] T 6c958599492f48b7a8e4946640073d5c P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.656301 30387 tablet_copy_client.cc:647] T 6c958599492f48b7a8e4946640073d5c P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.663518 30388 tablet_bootstrap.cc:654] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:10.671854 30387 tablet_copy_client.cc:523] T 6c958599492f48b7a8e4946640073d5c P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:10.672319 22483 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:10.673556 30387 tablet_bootstrap.cc:492] T 6c958599492f48b7a8e4946640073d5c P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:10.677261 30388 tablet_bootstrap.cc:492] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:10.677459 30388 ts_tablet_manager.cc:1367] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.000s
I20240502 07:59:10.678998 30388 raft_consensus.cc:348] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.679201 30388 raft_consensus.cc:374] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:10.679282 30388 raft_consensus.cc:729] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.679463 30388 consensus_queue.cc:260] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.682263 30388 ts_tablet_manager.cc:1398] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:59:10.682180 22221 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:10.685683 30389 tablet_bootstrap.cc:492] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:10.685833 30389 ts_tablet_manager.cc:1367] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.001s
I20240502 07:59:10.686580 30389 raft_consensus.cc:348] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.686769 30389 raft_consensus.cc:374] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:10.686841 30389 raft_consensus.cc:729] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.687016 30389 consensus_queue.cc:260] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.688731 30389 ts_tablet_manager.cc:1398] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 07:59:10.689554 24800 tablet_service.cc:1467] Processing CreateTablet for tablet b5cd6d896bd74c59848db52a235b78b2 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=3c759007a4b047acbe8f33665e1018fc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:10.689808 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5cd6d896bd74c59848db52a235b78b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.690523 24799 tablet_service.cc:1467] Processing CreateTablet for tablet 8fcb3fef170343d494d8dfbba33052d9 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=3c759007a4b047acbe8f33665e1018fc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:10.690737 24799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fcb3fef170343d494d8dfbba33052d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.692194 24930 tablet_service.cc:1467] Processing CreateTablet for tablet b5cd6d896bd74c59848db52a235b78b2 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=3c759007a4b047acbe8f33665e1018fc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:10.692476 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5cd6d896bd74c59848db52a235b78b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.693034 25063 tablet_service.cc:1467] Processing CreateTablet for tablet ff6bc97390164a37bcdaafd651c888cf (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=3c759007a4b047acbe8f33665e1018fc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:10.692847 25062 tablet_service.cc:1467] Processing CreateTablet for tablet 8fcb3fef170343d494d8dfbba33052d9 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=3c759007a4b047acbe8f33665e1018fc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:10.693271 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff6bc97390164a37bcdaafd651c888cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.694525 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fcb3fef170343d494d8dfbba33052d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.695425 25193 tablet_service.cc:1467] Processing CreateTablet for tablet b5cd6d896bd74c59848db52a235b78b2 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=3c759007a4b047acbe8f33665e1018fc]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:10.695689 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5cd6d896bd74c59848db52a235b78b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.695631 22351 tablet_service.cc:1467] Processing CreateTablet for tablet ff6bc97390164a37bcdaafd651c888cf (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=3c759007a4b047acbe8f33665e1018fc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:10.696321 30388 tablet_bootstrap.cc:492] T b5cd6d896bd74c59848db52a235b78b2 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:10.696583 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff6bc97390164a37bcdaafd651c888cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.698202 30389 tablet_bootstrap.cc:492] T b5cd6d896bd74c59848db52a235b78b2 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:10.697451 30388 tablet_bootstrap.cc:654] T b5cd6d896bd74c59848db52a235b78b2 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:10.699209 30389 tablet_bootstrap.cc:654] T b5cd6d896bd74c59848db52a235b78b2 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:10.700671 30387 tablet_bootstrap.cc:492] T 6c958599492f48b7a8e4946640073d5c P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:10.700969 30387 tablet_bootstrap.cc:492] T 6c958599492f48b7a8e4946640073d5c P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap complete.
I20240502 07:59:10.701141 30387 ts_tablet_manager.cc:1367] T 6c958599492f48b7a8e4946640073d5c P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20240502 07:59:10.701711 30387 raft_consensus.cc:348] T 6c958599492f48b7a8e4946640073d5c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:10.701833 30387 raft_consensus.cc:729] T 6c958599492f48b7a8e4946640073d5c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.701959 30387 consensus_queue.cc:260] T 6c958599492f48b7a8e4946640073d5c P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:10.704486 24930 tablet_service.cc:1467] Processing CreateTablet for tablet 8fcb3fef170343d494d8dfbba33052d9 (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=3c759007a4b047acbe8f33665e1018fc]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:10.704762 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fcb3fef170343d494d8dfbba33052d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.707703 30387 ts_tablet_manager.cc:1398] T 6c958599492f48b7a8e4946640073d5c P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20240502 07:59:10.704777 30398 tablet_bootstrap.cc:492] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:10.705019 24932 tablet_service.cc:1467] Processing CreateTablet for tablet ff6bc97390164a37bcdaafd651c888cf (DEFAULT_TABLE table=rebalancer_extra_table_36 [id=3c759007a4b047acbe8f33665e1018fc]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:10.708257 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff6bc97390164a37bcdaafd651c888cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.708350 30401 tablet_bootstrap.cc:492] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:10.709343 25104 tablet_copy_service.cc:342] P 227eec4b5f014d349661f6e0b39b9055: Request end of tablet copy session e51b10af49484aaa8f0267aebc1bf27f-6c958599492f48b7a8e4946640073d5c received from {username='slave'} at 127.15.129.131:57623
I20240502 07:59:10.709414 25104 tablet_copy_service.cc:433] P 227eec4b5f014d349661f6e0b39b9055: ending tablet copy session e51b10af49484aaa8f0267aebc1bf27f-6c958599492f48b7a8e4946640073d5c on tablet 6c958599492f48b7a8e4946640073d5c with peer e51b10af49484aaa8f0267aebc1bf27f
I20240502 07:59:10.709514 30401 tablet_bootstrap.cc:654] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:10.710819 30401 tablet_bootstrap.cc:492] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:10.710922 30401 ts_tablet_manager.cc:1367] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:10.711478 30401 raft_consensus.cc:348] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.711601 30401 raft_consensus.cc:374] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:10.711639 30398 tablet_bootstrap.cc:654] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:10.711642 30401 raft_consensus.cc:729] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.698786 30390 tablet_bootstrap.cc:492] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:10.711794 30401 consensus_queue.cc:260] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.712116 30401 ts_tablet_manager.cc:1398] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:10.713440 30390 tablet_bootstrap.cc:654] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:10.713798 30360 tablet_bootstrap.cc:492] T f1890c9d5c664106bacc97d440c9cbc2 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:10.714076 30360 tablet_bootstrap.cc:492] T f1890c9d5c664106bacc97d440c9cbc2 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap complete.
I20240502 07:59:10.714232 30360 ts_tablet_manager.cc:1367] T f1890c9d5c664106bacc97d440c9cbc2 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.123s user 0.028s sys 0.000s
I20240502 07:59:10.714623 30360 raft_consensus.cc:348] T f1890c9d5c664106bacc97d440c9cbc2 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:10.714730 30360 raft_consensus.cc:729] T f1890c9d5c664106bacc97d440c9cbc2 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.714888 30360 consensus_queue.cc:260] T f1890c9d5c664106bacc97d440c9cbc2 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:10.715562 30389 tablet_bootstrap.cc:492] T b5cd6d896bd74c59848db52a235b78b2 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:10.715636 30389 ts_tablet_manager.cc:1367] T b5cd6d896bd74c59848db52a235b78b2 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.001s
I20240502 07:59:10.716022 30389 raft_consensus.cc:348] T b5cd6d896bd74c59848db52a235b78b2 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:10.716118 30389 raft_consensus.cc:374] T b5cd6d896bd74c59848db52a235b78b2 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:10.716153 30389 raft_consensus.cc:729] T b5cd6d896bd74c59848db52a235b78b2 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.716286 30389 consensus_queue.cc:260] T b5cd6d896bd74c59848db52a235b78b2 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:10.716563 30389 ts_tablet_manager.cc:1398] T b5cd6d896bd74c59848db52a235b78b2 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:10.720947 30388 tablet_bootstrap.cc:492] T b5cd6d896bd74c59848db52a235b78b2 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:10.721048 30388 ts_tablet_manager.cc:1367] T b5cd6d896bd74c59848db52a235b78b2 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.002s
I20240502 07:59:10.722968 30388 raft_consensus.cc:348] T b5cd6d896bd74c59848db52a235b78b2 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:10.723140 30388 raft_consensus.cc:374] T b5cd6d896bd74c59848db52a235b78b2 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:10.723201 30388 raft_consensus.cc:729] T b5cd6d896bd74c59848db52a235b78b2 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.723367 30388 consensus_queue.cc:260] T b5cd6d896bd74c59848db52a235b78b2 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:10.723856 30388 ts_tablet_manager.cc:1398] T b5cd6d896bd74c59848db52a235b78b2 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:10.724619 30388 tablet_bootstrap.cc:492] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:10.725610 30388 tablet_bootstrap.cc:654] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:10.727453 30388 tablet_bootstrap.cc:492] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:10.727581 30388 ts_tablet_manager.cc:1367] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 07:59:10.729146 30388 raft_consensus.cc:348] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:10.729264 30388 raft_consensus.cc:374] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:10.729297 30388 raft_consensus.cc:729] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.729507 30388 consensus_queue.cc:260] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:10.729842 30388 ts_tablet_manager.cc:1398] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 07:59:10.718293 30360 ts_tablet_manager.cc:1398] T f1890c9d5c664106bacc97d440c9cbc2 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:10.732554 22523 tablet_copy_service.cc:342] P fabddb16c85944648a710ec52fc7244c: Request end of tablet copy session 43930dee0cd5498fa8e357a34dd71432-f1890c9d5c664106bacc97d440c9cbc2 received from {username='slave'} at 127.15.129.136:45143
I20240502 07:59:10.732636 22523 tablet_copy_service.cc:433] P fabddb16c85944648a710ec52fc7244c: ending tablet copy session 43930dee0cd5498fa8e357a34dd71432-f1890c9d5c664106bacc97d440c9cbc2 on tablet f1890c9d5c664106bacc97d440c9cbc2 with peer 43930dee0cd5498fa8e357a34dd71432
I20240502 07:59:10.734670 30398 tablet_bootstrap.cc:492] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:10.734781 30398 ts_tablet_manager.cc:1367] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20240502 07:59:10.735775 30398 raft_consensus.cc:348] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:10.735867 30398 raft_consensus.cc:374] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:10.735888 30398 raft_consensus.cc:729] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.736002 30398 consensus_queue.cc:260] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:10.736258 30398 ts_tablet_manager.cc:1398] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:10.736400 30398 tablet_bootstrap.cc:492] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:10.737138 30398 tablet_bootstrap.cc:654] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:10.746042 30398 tablet_bootstrap.cc:492] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:10.746151 30398 ts_tablet_manager.cc:1367] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 07:59:10.746619 30398 raft_consensus.cc:348] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:10.746734 30398 raft_consensus.cc:374] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:10.746759 30398 raft_consensus.cc:729] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.746870 30398 consensus_queue.cc:260] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:10.747131 30398 ts_tablet_manager.cc:1398] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:10.747372 30398 tablet_bootstrap.cc:492] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:10.747748 30390 tablet_bootstrap.cc:492] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:10.747857 30390 ts_tablet_manager.cc:1367] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.049s user 0.002s sys 0.000s
I20240502 07:59:10.748292 30398 tablet_bootstrap.cc:654] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:10.748276 30390 raft_consensus.cc:348] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.748378 30390 raft_consensus.cc:374] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:10.748435 30390 raft_consensus.cc:729] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.748574 30390 consensus_queue.cc:260] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.748832 30390 ts_tablet_manager.cc:1398] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:10.749652 30390 tablet_bootstrap.cc:492] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:10.751112 30390 tablet_bootstrap.cc:654] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:10.754787 30390 tablet_bootstrap.cc:492] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:10.754869 30390 ts_tablet_manager.cc:1367] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:10.755313 30390 raft_consensus.cc:348] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:10.755486 30390 raft_consensus.cc:374] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:10.755560 30390 raft_consensus.cc:729] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.755712 30390 consensus_queue.cc:260] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:10.756038 30390 ts_tablet_manager.cc:1398] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:10.756224 30390 tablet_bootstrap.cc:492] T ff6bc97390164a37bcdaafd651c888cf P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:10.760248 30390 tablet_bootstrap.cc:654] T ff6bc97390164a37bcdaafd651c888cf P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:10.761286 30390 tablet_bootstrap.cc:492] T ff6bc97390164a37bcdaafd651c888cf P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:10.761420 30390 ts_tablet_manager.cc:1367] T ff6bc97390164a37bcdaafd651c888cf P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:10.761893 30398 tablet_bootstrap.cc:492] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:10.761898 30390 raft_consensus.cc:348] T ff6bc97390164a37bcdaafd651c888cf P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.761981 30398 ts_tablet_manager.cc:1367] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 07:59:10.761991 30390 raft_consensus.cc:374] T ff6bc97390164a37bcdaafd651c888cf P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:10.762024 30390 raft_consensus.cc:729] T ff6bc97390164a37bcdaafd651c888cf P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.762143 30390 consensus_queue.cc:260] T ff6bc97390164a37bcdaafd651c888cf P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.762468 30398 raft_consensus.cc:348] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.762563 30398 raft_consensus.cc:374] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:10.762593 30398 raft_consensus.cc:729] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.762698 30398 consensus_queue.cc:260] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.762984 30398 ts_tablet_manager.cc:1398] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:10.763079 30390 ts_tablet_manager.cc:1398] T ff6bc97390164a37bcdaafd651c888cf P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:10.765281 28686 raft_consensus.cc:482] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:10.765390 28686 raft_consensus.cc:504] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:10.765758 28686 leader_election.cc:288] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:10.766127 25214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cd6d896bd74c59848db52a235b78b2" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:10.766275 25214 raft_consensus.cc:2454] T b5cd6d896bd74c59848db52a235b78b2 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:10.766611 24888 leader_election.cc:302] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:10.766638 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cd6d896bd74c59848db52a235b78b2" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:10.766767 24820 raft_consensus.cc:2454] T b5cd6d896bd74c59848db52a235b78b2 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:10.766784 28686 raft_consensus.cc:2781] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:10.766842 28686 raft_consensus.cc:482] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:10.766884 28686 raft_consensus.cc:3037] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:10.768363 28686 raft_consensus.cc:504] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:10.768747 28686 leader_election.cc:288] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:10.769310 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cd6d896bd74c59848db52a235b78b2" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:10.769460 24820 raft_consensus.cc:3037] T b5cd6d896bd74c59848db52a235b78b2 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:10.769457 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cd6d896bd74c59848db52a235b78b2" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:10.769565 25215 raft_consensus.cc:3037] T b5cd6d896bd74c59848db52a235b78b2 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:10.770531 25215 raft_consensus.cc:2454] T b5cd6d896bd74c59848db52a235b78b2 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:10.770531 24820 raft_consensus.cc:2454] T b5cd6d896bd74c59848db52a235b78b2 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:10.770838 24887 leader_election.cc:302] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:10.770923 30360 ts_tablet_manager.cc:902] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:10.771402 30360 tablet_copy_client.cc:308] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:59:10.771627 28686 raft_consensus.cc:2781] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:10.771719 28686 raft_consensus.cc:686] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:10.771858 28686 consensus_queue.cc:237] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:10.772415 22653 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet 2241a741bf85433b9fea512e9c530416 from peer 43930dee0cd5498fa8e357a34dd71432 ({username='slave'} at 127.15.129.136:34023)
I20240502 07:59:10.772522 22653 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet 2241a741bf85433b9fea512e9c530416 from peer 43930dee0cd5498fa8e357a34dd71432 at {username='slave'} at 127.15.129.136:34023: session id = 43930dee0cd5498fa8e357a34dd71432-2241a741bf85433b9fea512e9c530416
I20240502 07:59:10.773350 22653 tablet_copy_source_session.cc:205] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.774312 30360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2241a741bf85433b9fea512e9c530416. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.776202 30360 tablet_copy_client.cc:783] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.776367 30360 tablet_copy_client.cc:647] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.777412 22221 catalog_manager.cc:5484] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:10.779408 30360 tablet_copy_client.cc:523] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:10.780541 30360 tablet_bootstrap.cc:492] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:10.809342 30360 tablet_bootstrap.cc:492] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:10.809717 30360 tablet_bootstrap.cc:492] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap complete.
I20240502 07:59:10.809923 30360 ts_tablet_manager.cc:1367] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.029s user 0.027s sys 0.000s
I20240502 07:59:10.810460 30360 raft_consensus.cc:348] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:10.810564 30360 raft_consensus.cc:729] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.810673 30360 consensus_queue.cc:260] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:10.810966 30360 ts_tablet_manager.cc:1398] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:10.811446 22653 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session 43930dee0cd5498fa8e357a34dd71432-2241a741bf85433b9fea512e9c530416 received from {username='slave'} at 127.15.129.136:34023
I20240502 07:59:10.811524 22653 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session 43930dee0cd5498fa8e357a34dd71432-2241a741bf85433b9fea512e9c530416 on tablet 2241a741bf85433b9fea512e9c530416 with peer 43930dee0cd5498fa8e357a34dd71432
I20240502 07:59:10.843784 30405 raft_consensus.cc:482] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:10.843935 30405 raft_consensus.cc:504] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:10.844552 30405 leader_election.cc:288] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:10.844888 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fcb3fef170343d494d8dfbba33052d9" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:10.845039 25084 raft_consensus.cc:2454] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:10.845099 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fcb3fef170343d494d8dfbba33052d9" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:10.845222 24952 raft_consensus.cc:2454] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:10.845373 24754 leader_election.cc:302] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:10.845548 30405 raft_consensus.cc:2781] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:10.845595 30405 raft_consensus.cc:482] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:10.845640 30405 raft_consensus.cc:3037] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:10.845768 30458 ts_tablet_manager.cc:902] T 8477b961ce384edea078851122e196e9 P 4b4e7eeb7f3f42119c0363a79febc54b: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:10.846277 30458 tablet_copy_client.cc:308] T 8477b961ce384edea078851122e196e9 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:59:10.846680 22653 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet 8477b961ce384edea078851122e196e9 from peer 4b4e7eeb7f3f42119c0363a79febc54b ({username='slave'} at 127.15.129.133:55755)
I20240502 07:59:10.846793 22653 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet 8477b961ce384edea078851122e196e9 from peer 4b4e7eeb7f3f42119c0363a79febc54b at {username='slave'} at 127.15.129.133:55755: session id = 4b4e7eeb7f3f42119c0363a79febc54b-8477b961ce384edea078851122e196e9
I20240502 07:59:10.846864 30405 raft_consensus.cc:504] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:10.847198 30405 leader_election.cc:288] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:10.847658 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fcb3fef170343d494d8dfbba33052d9" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:10.847764 25084 raft_consensus.cc:3037] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:10.847826 22653 tablet_copy_source_session.cc:205] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:10.847895 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fcb3fef170343d494d8dfbba33052d9" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:10.847990 24952 raft_consensus.cc:3037] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:10.848690 30458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8477b961ce384edea078851122e196e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:10.848799 25084 raft_consensus.cc:2454] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:10.848999 24952 raft_consensus.cc:2454] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:10.849046 24754 leader_election.cc:302] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:10.849216 30405 raft_consensus.cc:2781] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:10.849296 30405 raft_consensus.cc:686] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:59:10.849452 30405 consensus_queue.cc:237] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:10.851119 30458 tablet_copy_client.cc:783] T 8477b961ce384edea078851122e196e9 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:10.851274 30458 tablet_copy_client.cc:647] T 8477b961ce384edea078851122e196e9 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:10.851805 22229 catalog_manager.cc:5484] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:10.854746 30458 tablet_copy_client.cc:523] T 8477b961ce384edea078851122e196e9 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:10.856146 30458 tablet_bootstrap.cc:492] T 8477b961ce384edea078851122e196e9 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:10.859488 30405 raft_consensus.cc:482] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:10.859665 30405 raft_consensus.cc:504] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.860668 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40da7efedda844eca55b346d041981f9" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:10.860710 30405 leader_election.cc:288] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:10.860805 25084 raft_consensus.cc:2454] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:10.860980 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40da7efedda844eca55b346d041981f9" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:10.861121 25213 raft_consensus.cc:2454] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:10.861421 24754 leader_election.cc:302] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:10.861843 30405 raft_consensus.cc:2781] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:10.861965 30405 raft_consensus.cc:482] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:10.862017 30405 raft_consensus.cc:3037] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:10.862799 30405 raft_consensus.cc:504] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.863445 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40da7efedda844eca55b346d041981f9" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:10.863457 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40da7efedda844eca55b346d041981f9" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:10.863562 25084 raft_consensus.cc:3037] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:10.863590 25213 raft_consensus.cc:3037] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:10.864264 25084 raft_consensus.cc:2454] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:10.864583 25213 raft_consensus.cc:2454] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:10.864589 30405 leader_election.cc:288] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:10.864606 24754 leader_election.cc:302] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:10.864763 30405 raft_consensus.cc:2781] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:10.864830 30405 raft_consensus.cc:686] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:59:10.864953 30405 consensus_queue.cc:237] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:10.867205 22223 catalog_manager.cc:5484] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:10.881004 30458 tablet_bootstrap.cc:492] T 8477b961ce384edea078851122e196e9 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:10.881295 30458 tablet_bootstrap.cc:492] T 8477b961ce384edea078851122e196e9 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap complete.
I20240502 07:59:10.881541 30458 ts_tablet_manager.cc:1367] T 8477b961ce384edea078851122e196e9 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.025s user 0.021s sys 0.003s
I20240502 07:59:10.882063 30458 raft_consensus.cc:348] T 8477b961ce384edea078851122e196e9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:10.882232 30458 raft_consensus.cc:729] T 8477b961ce384edea078851122e196e9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:10.882433 30458 consensus_queue.cc:260] T 8477b961ce384edea078851122e196e9 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:10.882768 30458 ts_tablet_manager.cc:1398] T 8477b961ce384edea078851122e196e9 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:10.883363 22653 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-8477b961ce384edea078851122e196e9 received from {username='slave'} at 127.15.129.133:55755
I20240502 07:59:10.883473 22653 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-8477b961ce384edea078851122e196e9 on tablet 8477b961ce384edea078851122e196e9 with peer 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:10.997170 22632 raft_consensus.cc:1203] T 6c958599492f48b7a8e4946640073d5c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:11.007793 22223 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:11.008671 30299 raft_consensus.cc:482] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:11.008781 30299 raft_consensus.cc:504] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:11.009239 30299 leader_election.cc:288] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:11.009644 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6bc97390164a37bcdaafd651c888cf" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:11.009706 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6bc97390164a37bcdaafd651c888cf" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:11.009799 25084 raft_consensus.cc:2454] T ff6bc97390164a37bcdaafd651c888cf P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:11.009819 24952 raft_consensus.cc:2454] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:11.010238 22305 leader_election.cc:302] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:11.010723 30299 raft_consensus.cc:2781] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:11.010782 30299 raft_consensus.cc:482] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:11.010826 30299 raft_consensus.cc:3037] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:11.011988 30299 raft_consensus.cc:504] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:11.012323 30299 leader_election.cc:288] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:11.012671 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6bc97390164a37bcdaafd651c888cf" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:11.012784 24952 raft_consensus.cc:3037] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:11.012823 25213 raft_consensus.cc:1261] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 4b4e7eeb7f3f42119c0363a79febc54b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:11.013121 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff6bc97390164a37bcdaafd651c888cf" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:11.013247 25084 raft_consensus.cc:3037] T ff6bc97390164a37bcdaafd651c888cf P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:11.013352 30405 consensus_queue.cc:1035] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:11.013890 24952 raft_consensus.cc:2454] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:11.014205 25084 raft_consensus.cc:2454] T ff6bc97390164a37bcdaafd651c888cf P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:11.014500 25083 raft_consensus.cc:1261] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 4b4e7eeb7f3f42119c0363a79febc54b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:11.014533 22305 leader_election.cc:302] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:11.014693 30299 raft_consensus.cc:2781] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:11.014768 30299 raft_consensus.cc:686] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:59:11.014899 30299 consensus_queue.cc:237] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:11.015630 29297 consensus_queue.cc:1035] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:11.016510 22223 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 40da7efedda844eca55b346d041981f9 (table rebalancer_dynamic_table [id=3af9bd84a0ce41f4a2b33787e975d248]). Expected version 1 got 0
I20240502 07:59:11.017282 30470 tablet.cc:1722] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:11.017863 30470 tablet.cc:1620] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:11.018122 30469 tablet.cc:1722] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:11.018499 30469 tablet.cc:1620] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:11.018788 22229 catalog_manager.cc:5484] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:11.021538 22223 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 40da7efedda844eca55b346d041981f9 (table rebalancer_dynamic_table [id=3af9bd84a0ce41f4a2b33787e975d248]). Expected version 1 got 0
I20240502 07:59:11.023049 30473 tablet.cc:1722] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:11.023406 30473 tablet.cc:1620] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:11.023748 22230 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=3af9bd84a0ce41f4a2b33787e975d248] alter complete (version 1)
I20240502 07:59:11.024344 22223 catalog_manager.cc:5576] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733) does not have the latest schema for tablet 40da7efedda844eca55b346d041981f9 (table rebalancer_dynamic_table [id=3af9bd84a0ce41f4a2b33787e975d248]). Expected version 1 got 0
I20240502 07:59:11.034806 25083 raft_consensus.cc:1203] T a0f577dafc0b4846a1a9e4dc54a5a687 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:11.041227 24932 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:11.046401 25063 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:11.063894 22613 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:11.066017 24799 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:11.067286 22763 raft_consensus.cc:1203] T 61690f2a278745aeace757c233e75b25 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:11.068621 25193 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:11.073680 22351 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:11.088451 22743 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:11.098383 22483 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:11.109882 25083 raft_consensus.cc:1203] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:11.135675 22230 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:11.143646 30469 tablet.cc:1722] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:11.144001 30469 tablet.cc:1620] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:11.145794 30470 tablet.cc:1722] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:11.146073 30470 tablet.cc:1620] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:11.149868 30473 tablet.cc:1722] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:11.150403 22230 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=3af9bd84a0ce41f4a2b33787e975d248] alter complete (version 2)
I20240502 07:59:11.150424 30473 tablet.cc:1620] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:11.155885 25213 raft_consensus.cc:1203] T f1890c9d5c664106bacc97d440c9cbc2 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:11.165459 25083 consensus_queue.cc:237] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:11.166698 22632 raft_consensus.cc:1261] T 6c958599492f48b7a8e4946640073d5c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:11.166743 25213 raft_consensus.cc:1261] T 6c958599492f48b7a8e4946640073d5c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:11.167076 30500 consensus_queue.cc:1035] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:11.167217 28464 consensus_queue.cc:1035] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:11.169430 28464 raft_consensus.cc:2932] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:11.169924 22632 raft_consensus.cc:2932] T 6c958599492f48b7a8e4946640073d5c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:11.171066 25213 raft_consensus.cc:2932] T 6c958599492f48b7a8e4946640073d5c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:11.172216 22223 catalog_manager.cc:5484] T 6c958599492f48b7a8e4946640073d5c P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: config changed from index 2 to 3, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) evicted. New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:11.175189 24799 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c958599492f48b7a8e4946640073d5c with delete_type TABLET_DATA_TOMBSTONED (TS 4b4e7eeb7f3f42119c0363a79febc54b not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:36878
I20240502 07:59:11.176187 30503 tablet_replica.cc:330] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:59:11.176335 30503 raft_consensus.cc:2229] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:11.176477 30503 raft_consensus.cc:2258] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:11.177021 30503 ts_tablet_manager.cc:1875] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:11.179957 30503 ts_tablet_manager.cc:1888] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:11.180042 30503 log.cc:1196] T 6c958599492f48b7a8e4946640073d5c P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/6c958599492f48b7a8e4946640073d5c
I20240502 07:59:11.180615 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet 6c958599492f48b7a8e4946640073d5c (table rebalancer_extra_table_4 [id=7fe3829782fb4a98912517ae49651ba5]) successfully deleted
I20240502 07:59:11.230078 28686 consensus_queue.cc:1035] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:11.234122 29729 consensus_queue.cc:1035] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:11.255659 22230 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:11.262835 30469 tablet.cc:1722] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:11.266672 30473 tablet.cc:1722] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:11.269454 30470 tablet.cc:1722] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:11.274073 22224 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=3af9bd84a0ce41f4a2b33787e975d248] alter complete (version 3)
I20240502 07:59:11.277175 25215 raft_consensus.cc:1203] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:11.309903 24932 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:11.313127 24821 raft_consensus.cc:1203] T 8477b961ce384edea078851122e196e9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:11.313270 24799 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:11.317400 25063 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:11.322343 25193 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:11.322577 22613 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:11.323670 22743 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:11.356184 22483 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:11.356184 30405 consensus_queue.cc:1035] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:11.361726 30405 consensus_queue.cc:1035] T 8fcb3fef170343d494d8dfbba33052d9 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:11.365815 22351 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:11.379761 22228 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:11.379942 22228 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:11.383445 22228 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 40da7efedda844eca55b346d041981f9
I20240502 07:59:11.384100 25193 tablet_service.cc:1514] Processing DeleteTablet for tablet 40da7efedda844eca55b346d041981f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:11 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:11.384378 24799 tablet_service.cc:1514] Processing DeleteTablet for tablet 40da7efedda844eca55b346d041981f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:11 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:59:11.384455 30580 tablet_replica.cc:330] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:11.384564 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet 40da7efedda844eca55b346d041981f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:11 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:11.384616 30580 raft_consensus.cc:2229] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:11.384737 30580 raft_consensus.cc:2258] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:11.384828 30503 tablet_replica.cc:330] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:59:11.384971 30503 raft_consensus.cc:2229] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:11.385169 30503 raft_consensus.cc:2258] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:11.385581 30580 ts_tablet_manager.cc:1875] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:11.385746 30503 ts_tablet_manager.cc:1875] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:11.388096 30503 ts_tablet_manager.cc:1888] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:11.388185 30503 log.cc:1196] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/40da7efedda844eca55b346d041981f9
I20240502 07:59:11.388430 30580 ts_tablet_manager.cc:1888] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:11.388548 30580 log.cc:1196] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/40da7efedda844eca55b346d041981f9
I20240502 07:59:11.388736 30503 ts_tablet_manager.cc:1909] T 40da7efedda844eca55b346d041981f9 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:59:11.388808 30580 ts_tablet_manager.cc:1909] T 40da7efedda844eca55b346d041981f9 P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:59:11.389497 30581 tablet_replica.cc:330] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:11.389528 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 40da7efedda844eca55b346d041981f9 (table rebalancer_dynamic_table_new_name [id=3af9bd84a0ce41f4a2b33787e975d248]) successfully deleted
I20240502 07:59:11.389657 30581 raft_consensus.cc:2229] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:11.389724 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet 40da7efedda844eca55b346d041981f9 (table rebalancer_dynamic_table_new_name [id=3af9bd84a0ce41f4a2b33787e975d248]) successfully deleted
I20240502 07:59:11.389879 30581 raft_consensus.cc:2258] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:11.390432 30581 ts_tablet_manager.cc:1875] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:11.394268 30581 ts_tablet_manager.cc:1888] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:11.394354 30581 log.cc:1196] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/40da7efedda844eca55b346d041981f9
I20240502 07:59:11.394677 30581 ts_tablet_manager.cc:1909] T 40da7efedda844eca55b346d041981f9 P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:59:11.395364 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet 40da7efedda844eca55b346d041981f9 (table rebalancer_dynamic_table_new_name [id=3af9bd84a0ce41f4a2b33787e975d248]) successfully deleted
I20240502 07:59:11.413704 22763 consensus_queue.cc:237] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:11.414988 25083 raft_consensus.cc:1261] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:11.415146 22632 raft_consensus.cc:1261] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:11.415546 30164 consensus_queue.cc:1035] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:11.415829 30164 consensus_queue.cc:1035] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:11.416360 30299 consensus_queue.cc:1035] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:11.417861 30164 raft_consensus.cc:2932] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:11.419440 29808 consensus_queue.cc:1035] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:11.420658 22632 raft_consensus.cc:2932] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:11.422399 22228 catalog_manager.cc:5484] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:11.425180 25083 raft_consensus.cc:2932] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:11.428956 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet 895f9a31785d4558acd3b64904a645ca with delete_type TABLET_DATA_TOMBSTONED (TS fabddb16c85944648a710ec52fc7244c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:11.429370 30608 tablet_replica.cc:330] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:11.429558 30608 raft_consensus.cc:2229] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:11.429858 30608 raft_consensus.cc:2258] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:11.430497 30608 ts_tablet_manager.cc:1875] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:11.432817 30608 ts_tablet_manager.cc:1888] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:11.433096 30608 log.cc:1196] T 895f9a31785d4558acd3b64904a645ca P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/895f9a31785d4558acd3b64904a645ca
I20240502 07:59:11.433737 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 895f9a31785d4558acd3b64904a645ca (table rebalance_test_table_15 [id=4e81cbd084694ac8b1db3f5574dde601]) successfully deleted
I20240502 07:59:11.485272 22230 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:11.492985 25193 tablet_service.cc:1467] Processing CreateTablet for tablet 72b46634b6bd42adbf7921db0752d7aa (DEFAULT_TABLE table=rebalancer_dynamic_table [id=7f29a73fe62b4eca9320c0015e929d3d]), partition=
I20240502 07:59:11.493310 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72b46634b6bd42adbf7921db0752d7aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:11.495930 24799 tablet_service.cc:1467] Processing CreateTablet for tablet 72b46634b6bd42adbf7921db0752d7aa (DEFAULT_TABLE table=rebalancer_dynamic_table [id=7f29a73fe62b4eca9320c0015e929d3d]), partition=
I20240502 07:59:11.496173 24799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72b46634b6bd42adbf7921db0752d7aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:11.498385 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 72b46634b6bd42adbf7921db0752d7aa (DEFAULT_TABLE table=rebalancer_dynamic_table [id=7f29a73fe62b4eca9320c0015e929d3d]), partition=
I20240502 07:59:11.498627 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72b46634b6bd42adbf7921db0752d7aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:11.500814 30647 tablet_bootstrap.cc:492] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:11.502241 30647 tablet_bootstrap.cc:654] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:11.506253 30646 tablet_bootstrap.cc:492] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:11.507257 30646 tablet_bootstrap.cc:654] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:11.508133 30649 tablet_bootstrap.cc:492] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:11.509092 30649 tablet_bootstrap.cc:654] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:11.510095 30646 tablet_bootstrap.cc:492] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:11.510206 30646 ts_tablet_manager.cc:1367] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:11.510803 30646 raft_consensus.cc:348] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:11.511058 30646 raft_consensus.cc:374] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:11.511159 30646 raft_consensus.cc:729] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:11.511341 30646 consensus_queue.cc:260] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:11.512732 30646 ts_tablet_manager.cc:1398] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:11.513659 30649 tablet_bootstrap.cc:492] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:11.513768 30649 ts_tablet_manager.cc:1367] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:11.514288 30649 raft_consensus.cc:348] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:11.514415 30649 raft_consensus.cc:374] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:11.514451 30649 raft_consensus.cc:729] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:11.514581 30649 consensus_queue.cc:260] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:11.514894 30649 ts_tablet_manager.cc:1398] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:11.516058 30647 tablet_bootstrap.cc:492] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:11.516224 30647 ts_tablet_manager.cc:1367] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.002s
I20240502 07:59:11.516785 30647 raft_consensus.cc:348] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:11.517228 30647 raft_consensus.cc:374] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:11.517339 30647 raft_consensus.cc:729] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:11.517525 30647 consensus_queue.cc:260] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:11.519156 30647 ts_tablet_manager.cc:1398] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:11.530584 22224 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:11.538718 25193 tablet_service.cc:1467] Processing CreateTablet for tablet efdd040f4d6748879b399081416b0c42 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=ed003ae80ddc4a70b20295b76fd5ee84]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:11.539048 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efdd040f4d6748879b399081416b0c42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:11.541635 25063 tablet_service.cc:1467] Processing CreateTablet for tablet efdd040f4d6748879b399081416b0c42 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=ed003ae80ddc4a70b20295b76fd5ee84]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:11.541909 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efdd040f4d6748879b399081416b0c42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:11.545456 24799 tablet_service.cc:1467] Processing CreateTablet for tablet efdd040f4d6748879b399081416b0c42 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=ed003ae80ddc4a70b20295b76fd5ee84]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:11.545467 24800 tablet_service.cc:1467] Processing CreateTablet for tablet bd77571fb07242f6b311c448fce0ae03 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=ed003ae80ddc4a70b20295b76fd5ee84]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:11.545782 24799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efdd040f4d6748879b399081416b0c42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:11.545938 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd77571fb07242f6b311c448fce0ae03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:11.547124 24932 tablet_service.cc:1467] Processing CreateTablet for tablet bd77571fb07242f6b311c448fce0ae03 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=ed003ae80ddc4a70b20295b76fd5ee84]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:11.547333 25193 tablet_service.cc:1467] Processing CreateTablet for tablet bd77571fb07242f6b311c448fce0ae03 (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=ed003ae80ddc4a70b20295b76fd5ee84]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:11.547422 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd77571fb07242f6b311c448fce0ae03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:11.547585 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd77571fb07242f6b311c448fce0ae03. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:11.548290 22351 tablet_service.cc:1467] Processing CreateTablet for tablet b2abf00f982e450fbe0124226789324a (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=ed003ae80ddc4a70b20295b76fd5ee84]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:11.548341 25194 tablet_service.cc:1467] Processing CreateTablet for tablet b2abf00f982e450fbe0124226789324a (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=ed003ae80ddc4a70b20295b76fd5ee84]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:11.548568 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2abf00f982e450fbe0124226789324a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:11.548574 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2abf00f982e450fbe0124226789324a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:11.551522 24800 tablet_service.cc:1467] Processing CreateTablet for tablet b2abf00f982e450fbe0124226789324a (DEFAULT_TABLE table=rebalancer_extra_table_38 [id=ed003ae80ddc4a70b20295b76fd5ee84]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:11.551755 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2abf00f982e450fbe0124226789324a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:11.554492 30649 tablet_bootstrap.cc:492] T bd77571fb07242f6b311c448fce0ae03 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:11.554623 30646 tablet_bootstrap.cc:492] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:11.555721 30646 tablet_bootstrap.cc:654] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:11.556072 30649 tablet_bootstrap.cc:654] T bd77571fb07242f6b311c448fce0ae03 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:11.556311 30647 tablet_bootstrap.cc:492] T efdd040f4d6748879b399081416b0c42 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:11.557039 30646 tablet_bootstrap.cc:492] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:11.557152 30646 ts_tablet_manager.cc:1367] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:11.557277 30647 tablet_bootstrap.cc:654] T efdd040f4d6748879b399081416b0c42 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:11.557633 30646 raft_consensus.cc:348] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:11.557808 30646 raft_consensus.cc:374] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:11.557871 30646 raft_consensus.cc:729] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:11.558060 30646 consensus_queue.cc:260] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:11.558240 30649 tablet_bootstrap.cc:492] T bd77571fb07242f6b311c448fce0ae03 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:11.558395 30649 ts_tablet_manager.cc:1367] T bd77571fb07242f6b311c448fce0ae03 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 07:59:11.558904 30649 raft_consensus.cc:348] T bd77571fb07242f6b311c448fce0ae03 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:11.559077 30649 raft_consensus.cc:374] T bd77571fb07242f6b311c448fce0ae03 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:11.559317 30649 raft_consensus.cc:729] T bd77571fb07242f6b311c448fce0ae03 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:11.560109 30649 consensus_queue.cc:260] T bd77571fb07242f6b311c448fce0ae03 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:11.560562 30649 ts_tablet_manager.cc:1398] T bd77571fb07242f6b311c448fce0ae03 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20240502 07:59:11.560868 30649 tablet_bootstrap.cc:492] T efdd040f4d6748879b399081416b0c42 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:11.561502 30653 tablet_bootstrap.cc:492] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:11.562234 30649 tablet_bootstrap.cc:654] T efdd040f4d6748879b399081416b0c42 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:11.562485 30652 tablet_bootstrap.cc:492] T bd77571fb07242f6b311c448fce0ae03 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:11.563779 30649 tablet_bootstrap.cc:492] T efdd040f4d6748879b399081416b0c42 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:11.563822 30646 ts_tablet_manager.cc:1398] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240502 07:59:11.563865 30649 ts_tablet_manager.cc:1367] T efdd040f4d6748879b399081416b0c42 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:11.564910 30649 raft_consensus.cc:348] T efdd040f4d6748879b399081416b0c42 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:11.565037 30649 raft_consensus.cc:374] T efdd040f4d6748879b399081416b0c42 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:11.565079 30649 raft_consensus.cc:729] T efdd040f4d6748879b399081416b0c42 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:11.565200 30649 consensus_queue.cc:260] T efdd040f4d6748879b399081416b0c42 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:11.565511 30649 ts_tablet_manager.cc:1398] T efdd040f4d6748879b399081416b0c42 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:11.565670 30649 tablet_bootstrap.cc:492] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:11.566550 30649 tablet_bootstrap.cc:654] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:11.566713 30647 tablet_bootstrap.cc:492] T efdd040f4d6748879b399081416b0c42 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:11.566797 30647 ts_tablet_manager.cc:1367] T efdd040f4d6748879b399081416b0c42 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20240502 07:59:11.567133 30646 tablet_bootstrap.cc:492] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:11.567292 30647 raft_consensus.cc:348] T efdd040f4d6748879b399081416b0c42 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:11.567396 30647 raft_consensus.cc:374] T efdd040f4d6748879b399081416b0c42 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:11.567433 30647 raft_consensus.cc:729] T efdd040f4d6748879b399081416b0c42 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:11.567559 30647 consensus_queue.cc:260] T efdd040f4d6748879b399081416b0c42 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:11.567801 30653 tablet_bootstrap.cc:654] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:11.568228 30647 ts_tablet_manager.cc:1398] T efdd040f4d6748879b399081416b0c42 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:11.568861 30649 tablet_bootstrap.cc:492] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:11.568940 30649 ts_tablet_manager.cc:1367] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:11.569463 30649 raft_consensus.cc:348] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:11.569571 30649 raft_consensus.cc:374] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:11.569613 30649 raft_consensus.cc:729] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:11.569721 30649 consensus_queue.cc:260] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:11.570006 30649 ts_tablet_manager.cc:1398] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:11.570425 30646 tablet_bootstrap.cc:654] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:11.571424 30652 tablet_bootstrap.cc:654] T bd77571fb07242f6b311c448fce0ae03 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:11.574682 30646 tablet_bootstrap.cc:492] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:11.574827 30646 ts_tablet_manager.cc:1367] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:59:11.576900 30646 raft_consensus.cc:348] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:11.577105 30646 raft_consensus.cc:374] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:11.577158 30646 raft_consensus.cc:729] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:11.577304 30646 consensus_queue.cc:260] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:11.578691 30646 ts_tablet_manager.cc:1398] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:11.578922 30646 tablet_bootstrap.cc:492] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:11.579285 30653 tablet_bootstrap.cc:492] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:11.579442 30653 ts_tablet_manager.cc:1367] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.001s
I20240502 07:59:11.579510 29609 raft_consensus.cc:482] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:11.579581 29609 raft_consensus.cc:504] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:11.580006 29609 leader_election.cc:288] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:11.579959 30653 raft_consensus.cc:348] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:11.580066 30653 raft_consensus.cc:374] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:11.580097 30653 raft_consensus.cc:729] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:11.580518 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efdd040f4d6748879b399081416b0c42" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:11.580662 24821 raft_consensus.cc:2454] T efdd040f4d6748879b399081416b0c42 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:11.580863 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efdd040f4d6748879b399081416b0c42" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:11.581004 25084 raft_consensus.cc:2454] T efdd040f4d6748879b399081416b0c42 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:11.581221 30653 consensus_queue.cc:260] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:11.581789 25150 leader_election.cc:302] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:11.581823 24932 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:11.581998 30653 ts_tablet_manager.cc:1398] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:11.583009 29609 raft_consensus.cc:2781] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:11.583070 29609 raft_consensus.cc:482] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:11.583102 29609 raft_consensus.cc:3037] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:11.584157 29609 raft_consensus.cc:504] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:11.584475 29609 leader_election.cc:288] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:11.586614 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efdd040f4d6748879b399081416b0c42" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:11.586735 24821 raft_consensus.cc:3037] T efdd040f4d6748879b399081416b0c42 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:11.587787 24821 raft_consensus.cc:2454] T efdd040f4d6748879b399081416b0c42 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:11.588275 25150 leader_election.cc:302] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:11.590212 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efdd040f4d6748879b399081416b0c42" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:11.590334 25084 raft_consensus.cc:3037] T efdd040f4d6748879b399081416b0c42 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:11.591281 25084 raft_consensus.cc:2454] T efdd040f4d6748879b399081416b0c42 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:11.591521 28593 raft_consensus.cc:2781] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:11.591657 28593 raft_consensus.cc:686] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:11.591778 28593 consensus_queue.cc:237] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:11.597035 22225 catalog_manager.cc:5484] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:11.597817 25063 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:11.598206 30646 tablet_bootstrap.cc:654] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:11.599789 30646 tablet_bootstrap.cc:492] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:11.599953 30646 ts_tablet_manager.cc:1367] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.002s
I20240502 07:59:11.600515 30646 raft_consensus.cc:348] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:11.600610 30646 raft_consensus.cc:374] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:11.600646 30646 raft_consensus.cc:729] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:11.600788 30646 consensus_queue.cc:260] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:11.601047 30646 ts_tablet_manager.cc:1398] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:11.601588 30652 tablet_bootstrap.cc:492] T bd77571fb07242f6b311c448fce0ae03 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:11.601693 30652 ts_tablet_manager.cc:1367] T bd77571fb07242f6b311c448fce0ae03 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.000s
I20240502 07:59:11.602211 30652 raft_consensus.cc:348] T bd77571fb07242f6b311c448fce0ae03 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:11.602320 30652 raft_consensus.cc:374] T bd77571fb07242f6b311c448fce0ae03 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:11.602351 30652 raft_consensus.cc:729] T bd77571fb07242f6b311c448fce0ae03 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:11.602474 30652 consensus_queue.cc:260] T bd77571fb07242f6b311c448fce0ae03 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:11.602756 30652 ts_tablet_manager.cc:1398] T bd77571fb07242f6b311c448fce0ae03 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:11.617051 25193 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:11.620141 22613 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:11.624845 22351 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:11.632002 22483 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:11.633834 22743 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:11.644037 29297 raft_consensus.cc:482] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:11.644160 29297 raft_consensus.cc:504] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:11.644582 29297 leader_election.cc:288] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:11.644915 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72b46634b6bd42adbf7921db0752d7aa" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:11.645061 25084 raft_consensus.cc:2454] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:11.645426 24754 leader_election.cc:302] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:11.646488 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72b46634b6bd42adbf7921db0752d7aa" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:11.646601 25215 raft_consensus.cc:2454] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:11.646817 29297 raft_consensus.cc:2781] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:11.646875 29297 raft_consensus.cc:482] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:11.646914 29297 raft_consensus.cc:3037] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:11.648092 29297 raft_consensus.cc:504] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:11.648415 29297 leader_election.cc:288] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:11.649137 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72b46634b6bd42adbf7921db0752d7aa" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:11.649250 25084 raft_consensus.cc:3037] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:11.650187 25084 raft_consensus.cc:2454] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:11.651856 24754 leader_election.cc:302] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:11.652678 29297 raft_consensus.cc:2781] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:11.652760 29297 raft_consensus.cc:686] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:59:11.652889 29297 consensus_queue.cc:237] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:11.653640 24800 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:11.656239 22230 catalog_manager.cc:5484] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:11.662132 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72b46634b6bd42adbf7921db0752d7aa" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:11.662278 25215 raft_consensus.cc:3037] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:11.663228 25215 raft_consensus.cc:2454] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:11.709352 29297 raft_consensus.cc:482] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:11.709481 29297 raft_consensus.cc:504] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:11.709894 29297 leader_election.cc:288] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:11.710227 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2abf00f982e450fbe0124226789324a" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:11.710366 25215 raft_consensus.cc:2454] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:11.710633 24757 leader_election.cc:302] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:11.710731 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2abf00f982e450fbe0124226789324a" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:11.710803 29297 raft_consensus.cc:2781] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:11.710850 29297 raft_consensus.cc:482] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:11.710888 29297 raft_consensus.cc:3037] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:11.710871 22372 raft_consensus.cc:2454] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:11.711961 29297 raft_consensus.cc:504] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:11.712342 29297 leader_election.cc:288] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:11.712685 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2abf00f982e450fbe0124226789324a" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:11.712838 25215 raft_consensus.cc:3037] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:11.712802 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2abf00f982e450fbe0124226789324a" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:11.712787 22500 consensus_queue.cc:237] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:11.712916 22372 raft_consensus.cc:3037] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:11.713899 22372 raft_consensus.cc:2454] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:11.713899 25215 raft_consensus.cc:2454] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:11.714186 24757 leader_election.cc:302] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:11.714411 25215 raft_consensus.cc:1261] T f1890c9d5c664106bacc97d440c9cbc2 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:11.714457 29297 raft_consensus.cc:2781] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:11.714534 29297 raft_consensus.cc:686] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:59:11.714543 22763 raft_consensus.cc:1261] T f1890c9d5c664106bacc97d440c9cbc2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:11.714656 29297 consensus_queue.cc:237] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:11.714756 30670 consensus_queue.cc:1035] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:11.716058 30670 consensus_queue.cc:1035] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:11.716979 22230 catalog_manager.cc:5484] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:11.717890 30135 raft_consensus.cc:2932] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:11.719782 22763 raft_consensus.cc:2932] T f1890c9d5c664106bacc97d440c9cbc2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:11.720065 25215 raft_consensus.cc:2932] T f1890c9d5c664106bacc97d440c9cbc2 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:11.720649 22225 catalog_manager.cc:5484] T f1890c9d5c664106bacc97d440c9cbc2 P fabddb16c85944648a710ec52fc7244c reported cstate change: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:11.728720 22613 tablet_service.cc:1514] Processing DeleteTablet for tablet f1890c9d5c664106bacc97d440c9cbc2 with delete_type TABLET_DATA_TOMBSTONED (TS e51b10af49484aaa8f0267aebc1bf27f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:11.729084 30699 tablet_replica.cc:330] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:11.729226 30699 raft_consensus.cc:2229] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:11.729463 30699 raft_consensus.cc:2258] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:11.729928 30699 ts_tablet_manager.cc:1875] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:11.732333 30699 ts_tablet_manager.cc:1888] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:11.732473 30699 log.cc:1196] T f1890c9d5c664106bacc97d440c9cbc2 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/f1890c9d5c664106bacc97d440c9cbc2
I20240502 07:59:11.733049 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet f1890c9d5c664106bacc97d440c9cbc2 (table rebalance_test_table_3 [id=2c887f37ddf349f3bbec2b1330f05fc7]) successfully deleted
I20240502 07:59:11.762104 22230 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:11.766222 25215 raft_consensus.cc:1261] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 4b4e7eeb7f3f42119c0363a79febc54b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:11.766235 25084 raft_consensus.cc:1261] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 4b4e7eeb7f3f42119c0363a79febc54b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:11.766633 30405 consensus_queue.cc:1035] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:11.766927 30405 consensus_queue.cc:1035] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:11.769996 22230 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 72b46634b6bd42adbf7921db0752d7aa (table rebalancer_dynamic_table [id=7f29a73fe62b4eca9320c0015e929d3d]). Expected version 1 got 0
I20240502 07:59:11.770838 22223 catalog_manager.cc:5576] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733) does not have the latest schema for tablet 72b46634b6bd42adbf7921db0752d7aa (table rebalancer_dynamic_table [id=7f29a73fe62b4eca9320c0015e929d3d]). Expected version 1 got 0
I20240502 07:59:11.771660 30726 tablet.cc:1722] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:11.772071 30726 tablet.cc:1620] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:11.771147 30725 tablet.cc:1722] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:11.772323 30725 tablet.cc:1620] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:11.775115 30727 tablet.cc:1722] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:11.776154 30727 tablet.cc:1620] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:11.776335 22223 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=7f29a73fe62b4eca9320c0015e929d3d] alter complete (version 1)
I20240502 07:59:11.776736 22230 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 72b46634b6bd42adbf7921db0752d7aa (table rebalancer_dynamic_table [id=7f29a73fe62b4eca9320c0015e929d3d]). Expected version 1 got 0
I20240502 07:59:11.785580 30673 raft_consensus.cc:482] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:11.785679 30673 raft_consensus.cc:504] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:11.786089 30673 leader_election.cc:288] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:11.786440 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd77571fb07242f6b311c448fce0ae03" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:11.786491 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd77571fb07242f6b311c448fce0ae03" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:11.786576 24820 raft_consensus.cc:2454] T bd77571fb07242f6b311c448fce0ae03 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:11.786635 24951 raft_consensus.cc:2454] T bd77571fb07242f6b311c448fce0ae03 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:11.786903 25150 leader_election.cc:302] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:11.787686 30673 raft_consensus.cc:2781] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:11.787751 30673 raft_consensus.cc:482] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:11.787793 30673 raft_consensus.cc:3037] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:11.788857 30673 raft_consensus.cc:504] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:11.789592 30673 leader_election.cc:288] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:11.790110 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd77571fb07242f6b311c448fce0ae03" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:11.790241 24951 raft_consensus.cc:3037] T bd77571fb07242f6b311c448fce0ae03 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:11.790522 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd77571fb07242f6b311c448fce0ae03" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:11.790632 24820 raft_consensus.cc:3037] T bd77571fb07242f6b311c448fce0ae03 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:11.791294 24951 raft_consensus.cc:2454] T bd77571fb07242f6b311c448fce0ae03 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:11.791607 24820 raft_consensus.cc:2454] T bd77571fb07242f6b311c448fce0ae03 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:11.791607 25151 leader_election.cc:302] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:11.791754 30673 raft_consensus.cc:2781] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:11.791857 30673 raft_consensus.cc:686] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:11.791993 30673 consensus_queue.cc:237] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:11.795271 22223 catalog_manager.cc:5484] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:11.841886 24932 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:11.850776 25063 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:11.852020 22351 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:11.852573 24800 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:11.857904 22613 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:11.857997 25193 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:11.879590 22483 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:11.881682 22223 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:11.897940 30725 tablet.cc:1722] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:11.898303 30725 tablet.cc:1620] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:11.898357 30727 tablet.cc:1722] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:11.898623 30727 tablet.cc:1620] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:11.901708 22743 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:11.904495 22228 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=7f29a73fe62b4eca9320c0015e929d3d] alter complete (version 2)
I20240502 07:59:11.906747 30726 tablet.cc:1722] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:11.907060 30726 tablet.cc:1620] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:11.974985 22632 consensus_queue.cc:237] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:11.976365 22763 raft_consensus.cc:1261] T a0f577dafc0b4846a1a9e4dc54a5a687 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:11.976438 25084 raft_consensus.cc:1261] T a0f577dafc0b4846a1a9e4dc54a5a687 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:11.976727 30113 consensus_queue.cc:1035] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:11.976904 29693 consensus_queue.cc:1035] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:11.980136 30113 raft_consensus.cc:2932] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:11.980862 22763 raft_consensus.cc:2932] T a0f577dafc0b4846a1a9e4dc54a5a687 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:11.984150 22223 catalog_manager.cc:5484] T a0f577dafc0b4846a1a9e4dc54a5a687 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:11.984952 25084 raft_consensus.cc:2932] T a0f577dafc0b4846a1a9e4dc54a5a687 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:11.992727 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet a0f577dafc0b4846a1a9e4dc54a5a687 with delete_type TABLET_DATA_TOMBSTONED (TS fabddb16c85944648a710ec52fc7244c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:11.993439 30783 tablet_replica.cc:330] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:11.993647 30783 raft_consensus.cc:2229] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:11.993813 30783 raft_consensus.cc:2258] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:11.994406 30783 ts_tablet_manager.cc:1875] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:11.996389 30783 ts_tablet_manager.cc:1888] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:11.996486 30783 log.cc:1196] T a0f577dafc0b4846a1a9e4dc54a5a687 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/a0f577dafc0b4846a1a9e4dc54a5a687
I20240502 07:59:11.997157 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet a0f577dafc0b4846a1a9e4dc54a5a687 (table rebalance_test_table_9 [id=9b69b771dcfb4a00a937ed9203245903]) successfully deleted
I20240502 07:59:12.006901 22228 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:12.015668 30725 tablet.cc:1722] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:12.021195 30727 tablet.cc:1722] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:12.023244 30726 tablet.cc:1722] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:12.033532 22225 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=7f29a73fe62b4eca9320c0015e929d3d] alter complete (version 3)
I20240502 07:59:12.083593 30673 consensus_queue.cc:1035] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:12.087368 30673 consensus_queue.cc:1035] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:12.134230 24932 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:12.134696 24800 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:12.136332 22221 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:12.136512 22221 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:12.139055 22221 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 72b46634b6bd42adbf7921db0752d7aa
I20240502 07:59:12.139668 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet 72b46634b6bd42adbf7921db0752d7aa with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:12 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:12.139811 24800 tablet_service.cc:1514] Processing DeleteTablet for tablet 72b46634b6bd42adbf7921db0752d7aa with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:12 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:59:12.140061 25193 tablet_service.cc:1514] Processing DeleteTablet for tablet 72b46634b6bd42adbf7921db0752d7aa with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:12 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:12.140092 25063 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:12.140318 30834 tablet_replica.cc:330] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:12.140375 30836 tablet_replica.cc:330] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:12.140717 30836 raft_consensus.cc:2229] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:12.140905 30836 raft_consensus.cc:2258] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:12.141405 30834 raft_consensus.cc:2229] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:12.141582 30834 raft_consensus.cc:2258] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:12.142446 30834 ts_tablet_manager.cc:1875] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:12.143110 30836 ts_tablet_manager.cc:1875] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:12.144989 30834 ts_tablet_manager.cc:1888] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:12.145054 30834 log.cc:1196] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/72b46634b6bd42adbf7921db0752d7aa
I20240502 07:59:12.145335 30834 ts_tablet_manager.cc:1909] T 72b46634b6bd42adbf7921db0752d7aa P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:59:12.145944 30836 ts_tablet_manager.cc:1888] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:12.146018 30836 log.cc:1196] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/72b46634b6bd42adbf7921db0752d7aa
I20240502 07:59:12.146266 30836 ts_tablet_manager.cc:1909] T 72b46634b6bd42adbf7921db0752d7aa P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:59:12.146245 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet 72b46634b6bd42adbf7921db0752d7aa (table rebalancer_dynamic_table_new_name [id=7f29a73fe62b4eca9320c0015e929d3d]) successfully deleted
I20240502 07:59:12.146921 30835 tablet_replica.cc:330] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:59:12.146921 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 72b46634b6bd42adbf7921db0752d7aa (table rebalancer_dynamic_table_new_name [id=7f29a73fe62b4eca9320c0015e929d3d]) successfully deleted
I20240502 07:59:12.147153 30835 raft_consensus.cc:2229] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:12.147418 30835 raft_consensus.cc:2258] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:12.151551 22613 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:12.153723 25193 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:12.154348 22351 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:12.161460 30835 ts_tablet_manager.cc:1875] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:12.165246 30835 ts_tablet_manager.cc:1888] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:12.165514 30835 log.cc:1196] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/72b46634b6bd42adbf7921db0752d7aa
I20240502 07:59:12.165880 30835 ts_tablet_manager.cc:1909] T 72b46634b6bd42adbf7921db0752d7aa P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:59:12.166055 22743 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:12.167294 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet 72b46634b6bd42adbf7921db0752d7aa (table rebalancer_dynamic_table_new_name [id=7f29a73fe62b4eca9320c0015e929d3d]) successfully deleted
I20240502 07:59:12.187160 22483 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:12.229730 29297 consensus_queue.cc:1035] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:12.240154 29297 consensus_queue.cc:1035] T b2abf00f982e450fbe0124226789324a P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:12.240980 22221 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:12.244836 22372 consensus_queue.cc:237] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:12.246623 25084 raft_consensus.cc:1261] T 15f70916ad6248e2a24334020df2c55e P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:12.247085 30299 consensus_queue.cc:1035] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:12.247797 22351 tablet_service.cc:1467] Processing CreateTablet for tablet 5b189f1a92db4c3aaaed36f636115949 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=264131b83d524000b0c7328f18e5a6e4]), partition=
I20240502 07:59:12.247830 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 5b189f1a92db4c3aaaed36f636115949 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=264131b83d524000b0c7328f18e5a6e4]), partition=
I20240502 07:59:12.247931 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 5b189f1a92db4c3aaaed36f636115949 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=264131b83d524000b0c7328f18e5a6e4]), partition=
I20240502 07:59:12.248032 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b189f1a92db4c3aaaed36f636115949. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:12.248081 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b189f1a92db4c3aaaed36f636115949. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:12.248132 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b189f1a92db4c3aaaed36f636115949. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:12.250597 30858 tablet_bootstrap.cc:492] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:12.251663 30858 tablet_bootstrap.cc:654] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:12.251900 30857 tablet_bootstrap.cc:492] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:12.252136 22632 raft_consensus.cc:1261] T 15f70916ad6248e2a24334020df2c55e P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:12.252971 30857 tablet_bootstrap.cc:654] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:12.252781 30299 consensus_queue.cc:1035] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:12.254858 30299 raft_consensus.cc:2932] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:12.255028 30856 tablet_bootstrap.cc:492] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:12.256076 30856 tablet_bootstrap.cc:654] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:12.257171 25083 raft_consensus.cc:2932] T 15f70916ad6248e2a24334020df2c55e P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:12.257598 30858 tablet_bootstrap.cc:492] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:12.257688 30858 ts_tablet_manager.cc:1367] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:12.258100 22228 catalog_manager.cc:5484] T 15f70916ad6248e2a24334020df2c55e P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:12.258170 30858 raft_consensus.cc:348] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.258280 30858 raft_consensus.cc:374] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:12.258324 30858 raft_consensus.cc:729] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:12.258494 30858 consensus_queue.cc:260] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.258879 30858 ts_tablet_manager.cc:1398] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:12.260298 30857 tablet_bootstrap.cc:492] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:12.261849 30857 ts_tablet_manager.cc:1367] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 07:59:12.262952 30857 raft_consensus.cc:348] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.263171 30857 raft_consensus.cc:374] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:12.263221 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet 15f70916ad6248e2a24334020df2c55e with delete_type TABLET_DATA_TOMBSTONED (TS fabddb16c85944648a710ec52fc7244c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:12.263231 30857 raft_consensus.cc:729] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:12.263454 30783 tablet_replica.cc:330] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:12.263417 30857 consensus_queue.cc:260] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.263350 22632 raft_consensus.cc:2932] T 15f70916ad6248e2a24334020df2c55e P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:12.263598 30783 raft_consensus.cc:2229] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:12.263732 30783 raft_consensus.cc:2258] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:12.264240 30857 ts_tablet_manager.cc:1398] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:12.264286 30783 ts_tablet_manager.cc:1875] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:12.264729 30856 tablet_bootstrap.cc:492] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:12.264827 30856 ts_tablet_manager.cc:1367] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 07:59:12.265365 30856 raft_consensus.cc:348] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.265483 30856 raft_consensus.cc:374] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:12.265517 30856 raft_consensus.cc:729] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:12.265643 30856 consensus_queue.cc:260] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.266103 30856 ts_tablet_manager.cc:1398] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240502 07:59:12.266623 30783 ts_tablet_manager.cc:1888] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:12.266695 30783 log.cc:1196] T 15f70916ad6248e2a24334020df2c55e P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/15f70916ad6248e2a24334020df2c55e
I20240502 07:59:12.267591 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 15f70916ad6248e2a24334020df2c55e (table rebalance_test_table_17 [id=1ec511fc66814cefa73a9b7acff4eea9]) successfully deleted
I20240502 07:59:12.281592 30135 raft_consensus.cc:482] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:12.281688 30135 raft_consensus.cc:504] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.282025 30135 leader_election.cc:288] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:12.282358 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b189f1a92db4c3aaaed36f636115949" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:12.282500 22370 raft_consensus.cc:2454] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:12.282711 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b189f1a92db4c3aaaed36f636115949" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:12.282836 24821 raft_consensus.cc:2454] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:12.283107 22438 leader_election.cc:302] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:12.283280 30135 raft_consensus.cc:2781] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:12.283329 30135 raft_consensus.cc:482] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:12.283355 30135 raft_consensus.cc:3037] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:12.284334 30135 raft_consensus.cc:504] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.284675 30135 leader_election.cc:288] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:12.285009 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b189f1a92db4c3aaaed36f636115949" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:12.285125 22370 raft_consensus.cc:3037] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:12.285447 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b189f1a92db4c3aaaed36f636115949" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:12.285547 24821 raft_consensus.cc:3037] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:12.286557 24821 raft_consensus.cc:2454] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:12.287281 30673 consensus_queue.cc:1035] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:12.287773 22438 leader_election.cc:302] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:12.289350 22370 raft_consensus.cc:2454] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:12.289602 30135 raft_consensus.cc:2781] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:12.289681 30135 raft_consensus.cc:686] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:59:12.289803 30135 consensus_queue.cc:237] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.292088 22228 catalog_manager.cc:5484] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:12.292176 30671 consensus_queue.cc:1035] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:12.359092 22228 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_40"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:12.366132 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 8aa4489eaac144c7926c6be2f199c46d (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=b12bf8ee0f3a4224a3269ebae6c91866]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:12.366209 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 8aa4489eaac144c7926c6be2f199c46d (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=b12bf8ee0f3a4224a3269ebae6c91866]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:12.366452 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aa4489eaac144c7926c6be2f199c46d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:12.366461 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aa4489eaac144c7926c6be2f199c46d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:12.367280 25194 tablet_service.cc:1467] Processing CreateTablet for tablet 038a695352ca4da58afaeb63a7785866 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=b12bf8ee0f3a4224a3269ebae6c91866]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:12.367568 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 038a695352ca4da58afaeb63a7785866. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:12.368724 30856 tablet_bootstrap.cc:492] T 8aa4489eaac144c7926c6be2f199c46d P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:12.370028 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 038a695352ca4da58afaeb63a7785866 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=b12bf8ee0f3a4224a3269ebae6c91866]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:12.370196 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 62c4b29fb02246c69d62765889b533ca (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=b12bf8ee0f3a4224a3269ebae6c91866]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:12.370303 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 038a695352ca4da58afaeb63a7785866. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:12.370483 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62c4b29fb02246c69d62765889b533ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:12.370999 30925 tablet_bootstrap.cc:492] T 8aa4489eaac144c7926c6be2f199c46d P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:12.371973 30925 tablet_bootstrap.cc:654] T 8aa4489eaac144c7926c6be2f199c46d P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:12.370028 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 62c4b29fb02246c69d62765889b533ca (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=b12bf8ee0f3a4224a3269ebae6c91866]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:12.372985 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62c4b29fb02246c69d62765889b533ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:12.373154 30925 tablet_bootstrap.cc:492] T 8aa4489eaac144c7926c6be2f199c46d P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:12.373250 30925 ts_tablet_manager.cc:1367] T 8aa4489eaac144c7926c6be2f199c46d P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:12.367280 25193 tablet_service.cc:1467] Processing CreateTablet for tablet 8aa4489eaac144c7926c6be2f199c46d (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=b12bf8ee0f3a4224a3269ebae6c91866]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:12.373580 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aa4489eaac144c7926c6be2f199c46d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:12.373711 30925 raft_consensus.cc:348] T 8aa4489eaac144c7926c6be2f199c46d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:12.373822 30925 raft_consensus.cc:374] T 8aa4489eaac144c7926c6be2f199c46d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:12.373853 30925 raft_consensus.cc:729] T 8aa4489eaac144c7926c6be2f199c46d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:12.373975 30925 consensus_queue.cc:260] T 8aa4489eaac144c7926c6be2f199c46d P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:12.374284 30925 ts_tablet_manager.cc:1398] T 8aa4489eaac144c7926c6be2f199c46d P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:12.376205 22482 tablet_service.cc:1467] Processing CreateTablet for tablet 62c4b29fb02246c69d62765889b533ca (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=b12bf8ee0f3a4224a3269ebae6c91866]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:12.376479 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62c4b29fb02246c69d62765889b533ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:12.377126 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 038a695352ca4da58afaeb63a7785866 (DEFAULT_TABLE table=rebalancer_extra_table_40 [id=b12bf8ee0f3a4224a3269ebae6c91866]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:12.377653 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 038a695352ca4da58afaeb63a7785866. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:12.378286 30856 tablet_bootstrap.cc:654] T 8aa4489eaac144c7926c6be2f199c46d P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:12.379526 30857 tablet_bootstrap.cc:492] T 62c4b29fb02246c69d62765889b533ca P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:12.380398 30857 tablet_bootstrap.cc:654] T 62c4b29fb02246c69d62765889b533ca P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:12.382671 30926 tablet_bootstrap.cc:492] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:12.385975 30926 tablet_bootstrap.cc:654] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:12.386224 30927 tablet_bootstrap.cc:492] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:12.386531 30856 tablet_bootstrap.cc:492] T 8aa4489eaac144c7926c6be2f199c46d P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:12.386626 30856 ts_tablet_manager.cc:1367] T 8aa4489eaac144c7926c6be2f199c46d P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.001s
I20240502 07:59:12.387187 30856 raft_consensus.cc:348] T 8aa4489eaac144c7926c6be2f199c46d P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:12.387312 30856 raft_consensus.cc:374] T 8aa4489eaac144c7926c6be2f199c46d P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:12.387346 30856 raft_consensus.cc:729] T 8aa4489eaac144c7926c6be2f199c46d P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:12.387462 30856 consensus_queue.cc:260] T 8aa4489eaac144c7926c6be2f199c46d P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:12.387547 30926 tablet_bootstrap.cc:492] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:12.387641 30926 ts_tablet_manager.cc:1367] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20240502 07:59:12.387761 30856 ts_tablet_manager.cc:1398] T 8aa4489eaac144c7926c6be2f199c46d P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:12.387812 30928 tablet_bootstrap.cc:492] T 62c4b29fb02246c69d62765889b533ca P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:12.388105 30926 raft_consensus.cc:348] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:12.388206 30926 raft_consensus.cc:374] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:12.388252 30926 raft_consensus.cc:729] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:12.388370 30926 consensus_queue.cc:260] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:12.388653 30926 ts_tablet_manager.cc:1398] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:12.388819 30926 tablet_bootstrap.cc:492] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:12.389806 30926 tablet_bootstrap.cc:654] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:12.391033 30857 tablet_bootstrap.cc:492] T 62c4b29fb02246c69d62765889b533ca P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:12.391116 30857 ts_tablet_manager.cc:1367] T 62c4b29fb02246c69d62765889b533ca P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 07:59:12.391594 30857 raft_consensus.cc:348] T 62c4b29fb02246c69d62765889b533ca P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:12.391713 30857 raft_consensus.cc:374] T 62c4b29fb02246c69d62765889b533ca P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:12.391748 30857 raft_consensus.cc:729] T 62c4b29fb02246c69d62765889b533ca P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:12.391861 30857 consensus_queue.cc:260] T 62c4b29fb02246c69d62765889b533ca P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:12.392184 30857 ts_tablet_manager.cc:1398] T 62c4b29fb02246c69d62765889b533ca P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:12.392758 30857 tablet_bootstrap.cc:492] T 038a695352ca4da58afaeb63a7785866 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:12.393702 30857 tablet_bootstrap.cc:654] T 038a695352ca4da58afaeb63a7785866 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:12.393707 30928 tablet_bootstrap.cc:654] T 62c4b29fb02246c69d62765889b533ca P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:12.393782 30926 tablet_bootstrap.cc:492] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:12.393855 30926 ts_tablet_manager.cc:1367] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:12.393877 30927 tablet_bootstrap.cc:654] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:12.394308 30926 raft_consensus.cc:348] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:12.394408 30926 raft_consensus.cc:374] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:12.394439 30926 raft_consensus.cc:729] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:12.394567 30926 consensus_queue.cc:260] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:12.394860 30926 ts_tablet_manager.cc:1398] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:12.398893 30857 tablet_bootstrap.cc:492] T 038a695352ca4da58afaeb63a7785866 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:12.398999 30857 ts_tablet_manager.cc:1367] T 038a695352ca4da58afaeb63a7785866 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:12.399648 30857 raft_consensus.cc:348] T 038a695352ca4da58afaeb63a7785866 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:12.399796 30857 raft_consensus.cc:374] T 038a695352ca4da58afaeb63a7785866 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:12.399875 30857 raft_consensus.cc:729] T 038a695352ca4da58afaeb63a7785866 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:12.400022 30857 consensus_queue.cc:260] T 038a695352ca4da58afaeb63a7785866 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:12.400457 30857 ts_tablet_manager.cc:1398] T 038a695352ca4da58afaeb63a7785866 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:12.400468 22227 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:12.401393 30927 tablet_bootstrap.cc:492] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:12.401589 30927 ts_tablet_manager.cc:1367] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20240502 07:59:12.402135 30927 raft_consensus.cc:348] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:12.402303 30927 raft_consensus.cc:374] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:12.402361 30927 raft_consensus.cc:729] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:12.402498 30927 consensus_queue.cc:260] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:12.402956 30927 ts_tablet_manager.cc:1398] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:12.404124 30927 tablet_bootstrap.cc:492] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:12.405759 24820 raft_consensus.cc:1261] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:12.406128 22370 raft_consensus.cc:1261] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:12.407649 30670 consensus_queue.cc:1035] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:12.407750 30927 tablet_bootstrap.cc:654] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:12.408335 30670 consensus_queue.cc:1035] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:12.411893 22227 catalog_manager.cc:5576] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535) does not have the latest schema for tablet 5b189f1a92db4c3aaaed36f636115949 (table rebalancer_dynamic_table [id=264131b83d524000b0c7328f18e5a6e4]). Expected version 1 got 0
I20240502 07:59:12.412998 22229 catalog_manager.cc:5576] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661) does not have the latest schema for tablet 5b189f1a92db4c3aaaed36f636115949 (table rebalancer_dynamic_table [id=264131b83d524000b0c7328f18e5a6e4]). Expected version 1 got 0
I20240502 07:59:12.412330 30940 tablet.cc:1722] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:12.413707 30940 tablet.cc:1620] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c: MemRowSet was empty: no flush needed.
I20240502 07:59:12.416642 30927 tablet_bootstrap.cc:492] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:12.416743 30927 ts_tablet_manager.cc:1367] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 07:59:12.417191 30927 raft_consensus.cc:348] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:12.417300 30927 raft_consensus.cc:374] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:12.417358 30927 raft_consensus.cc:729] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:12.417507 30927 consensus_queue.cc:260] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:12.419142 30725 tablet.cc:1722] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:12.419445 30725 tablet.cc:1620] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:12.420161 30927 ts_tablet_manager.cc:1398] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:12.422279 30950 tablet.cc:1722] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:12.422583 30950 tablet.cc:1620] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:12.424674 22229 catalog_manager.cc:5576] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535) does not have the latest schema for tablet 5b189f1a92db4c3aaaed36f636115949 (table rebalancer_dynamic_table [id=264131b83d524000b0c7328f18e5a6e4]). Expected version 1 got 0
I20240502 07:59:12.425490 22227 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=264131b83d524000b0c7328f18e5a6e4] alter complete (version 1)
I20240502 07:59:12.434509 30928 tablet_bootstrap.cc:492] T 62c4b29fb02246c69d62765889b533ca P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:12.434639 30928 ts_tablet_manager.cc:1367] T 62c4b29fb02246c69d62765889b533ca P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.048s user 0.002s sys 0.000s
I20240502 07:59:12.435166 30928 raft_consensus.cc:348] T 62c4b29fb02246c69d62765889b533ca P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:12.435289 30928 raft_consensus.cc:374] T 62c4b29fb02246c69d62765889b533ca P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:12.435329 30928 raft_consensus.cc:729] T 62c4b29fb02246c69d62765889b533ca P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:12.435456 30928 consensus_queue.cc:260] T 62c4b29fb02246c69d62765889b533ca P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:12.435755 30928 ts_tablet_manager.cc:1398] T 62c4b29fb02246c69d62765889b533ca P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:12.445119 24932 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:12.447917 25063 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:12.449718 24800 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:12.452239 25193 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:12.458227 30673 raft_consensus.cc:482] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:12.458336 30673 raft_consensus.cc:504] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:12.458706 30673 leader_election.cc:288] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:12.459386 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aa4489eaac144c7926c6be2f199c46d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:12.459532 25084 raft_consensus.cc:2454] T 8aa4489eaac144c7926c6be2f199c46d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:12.459854 25148 leader_election.cc:302] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:12.461448 30673 raft_consensus.cc:2781] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:12.461515 30673 raft_consensus.cc:482] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:12.461552 30673 raft_consensus.cc:3037] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:12.462586 30673 raft_consensus.cc:504] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:12.462906 30673 leader_election.cc:288] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:12.463454 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aa4489eaac144c7926c6be2f199c46d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:12.463572 25083 raft_consensus.cc:3037] T 8aa4489eaac144c7926c6be2f199c46d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:12.464545 25083 raft_consensus.cc:2454] T 8aa4489eaac144c7926c6be2f199c46d P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:12.468715 25148 leader_election.cc:302] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:12.469117 30671 raft_consensus.cc:2781] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:12.469199 30671 raft_consensus.cc:686] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:12.469712 30671 consensus_queue.cc:237] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:12.472126 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aa4489eaac144c7926c6be2f199c46d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:12.472294 24821 raft_consensus.cc:2454] T 8aa4489eaac144c7926c6be2f199c46d P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:12.472498 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aa4489eaac144c7926c6be2f199c46d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:12.472579 24820 raft_consensus.cc:3037] T 8aa4489eaac144c7926c6be2f199c46d P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:12.472560 22230 catalog_manager.cc:5484] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:12.473764 24820 raft_consensus.cc:2454] T 8aa4489eaac144c7926c6be2f199c46d P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:12.492563 22613 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:12.499053 22483 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:12.503440 22351 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:12.505610 22745 tablet_service.cc:1429] Tablet server has 7 leaders and 0 scanners
I20240502 07:59:12.531206 22230 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:12.538358 30940 tablet.cc:1722] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:12.538758 30940 tablet.cc:1620] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c: MemRowSet was empty: no flush needed.
I20240502 07:59:12.540061 30950 tablet.cc:1722] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:12.540385 30950 tablet.cc:1620] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:12.543640 30725 tablet.cc:1722] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:12.543964 30725 tablet.cc:1620] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:12.546851 22227 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=264131b83d524000b0c7328f18e5a6e4] alter complete (version 2)
I20240502 07:59:12.567319 30598 raft_consensus.cc:482] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:12.567457 30598 raft_consensus.cc:504] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:12.567970 30598 leader_election.cc:288] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:12.568354 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62c4b29fb02246c69d62765889b533ca" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:12.568508 24951 raft_consensus.cc:2454] T 62c4b29fb02246c69d62765889b533ca P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:12.568523 22500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62c4b29fb02246c69d62765889b533ca" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:12.568642 22500 raft_consensus.cc:2454] T 62c4b29fb02246c69d62765889b533ca P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:12.568848 22701 leader_election.cc:302] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:12.569173 30598 raft_consensus.cc:2781] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:12.569231 30598 raft_consensus.cc:482] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:12.569264 30598 raft_consensus.cc:3037] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:12.570436 30598 raft_consensus.cc:504] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:12.570819 30598 leader_election.cc:288] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:12.571537 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62c4b29fb02246c69d62765889b533ca" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:12.571601 22500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62c4b29fb02246c69d62765889b533ca" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:12.571671 24951 raft_consensus.cc:3037] T 62c4b29fb02246c69d62765889b533ca P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:12.571746 22500 raft_consensus.cc:3037] T 62c4b29fb02246c69d62765889b533ca P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:12.572659 24951 raft_consensus.cc:2454] T 62c4b29fb02246c69d62765889b533ca P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:12.572762 22500 raft_consensus.cc:2454] T 62c4b29fb02246c69d62765889b533ca P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:12.572928 22701 leader_election.cc:302] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:12.573108 30598 raft_consensus.cc:2781] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:12.573289 30598 raft_consensus.cc:686] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:59:12.573501 30598 consensus_queue.cc:237] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:12.575882 22229 catalog_manager.cc:5484] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:12.587541 22632 consensus_queue.cc:237] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:12.588511 30598 raft_consensus.cc:482] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:12.588624 30598 raft_consensus.cc:504] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:12.588812 22763 raft_consensus.cc:1261] T 61690f2a278745aeace757c233e75b25 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:12.589093 30598 leader_election.cc:288] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:12.589111 25215 raft_consensus.cc:1261] T 61690f2a278745aeace757c233e75b25 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:12.589212 30776 consensus_queue.cc:1035] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:12.589701 22500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "038a695352ca4da58afaeb63a7785866" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:12.589840 22500 raft_consensus.cc:2454] T 038a695352ca4da58afaeb63a7785866 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:12.589941 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "038a695352ca4da58afaeb63a7785866" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:12.590052 25215 raft_consensus.cc:2454] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:12.590196 22698 leader_election.cc:302] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:12.590581 30966 raft_consensus.cc:2781] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:12.590554 29693 consensus_queue.cc:1035] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:12.590725 30966 raft_consensus.cc:482] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:12.590786 30966 raft_consensus.cc:3037] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:12.591328 30776 raft_consensus.cc:2932] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:12.591933 22763 raft_consensus.cc:2932] T 61690f2a278745aeace757c233e75b25 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:12.592494 30966 raft_consensus.cc:504] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:12.593272 30966 leader_election.cc:288] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:12.593636 22500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "038a695352ca4da58afaeb63a7785866" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:12.593799 22500 raft_consensus.cc:3037] T 038a695352ca4da58afaeb63a7785866 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:12.594318 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "038a695352ca4da58afaeb63a7785866" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:12.594419 25213 raft_consensus.cc:3037] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:12.594527 22229 catalog_manager.cc:5484] T 61690f2a278745aeace757c233e75b25 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:12.594818 22500 raft_consensus.cc:2454] T 038a695352ca4da58afaeb63a7785866 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:12.595427 25213 raft_consensus.cc:2454] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:12.595633 22698 leader_election.cc:302] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:12.595793 30598 raft_consensus.cc:2781] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:12.595860 30598 raft_consensus.cc:686] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:59:12.595978 30598 consensus_queue.cc:237] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:12.596570 25215 raft_consensus.cc:2932] T 61690f2a278745aeace757c233e75b25 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:12.598062 24932 tablet_service.cc:1514] Processing DeleteTablet for tablet 61690f2a278745aeace757c233e75b25 with delete_type TABLET_DATA_TOMBSTONED (TS 1aecef81733c47b3b468504934357231 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:12.598423 30977 tablet_replica.cc:330] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:12.598634 30977 raft_consensus.cc:2229] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:12.598767 30977 raft_consensus.cc:2258] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:12.599200 30977 ts_tablet_manager.cc:1875] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:12.601063 22229 catalog_manager.cc:5484] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:12.601440 30977 ts_tablet_manager.cc:1888] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:12.601503 30977 log.cc:1196] T 61690f2a278745aeace757c233e75b25 P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/61690f2a278745aeace757c233e75b25
I20240502 07:59:12.602036 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet 61690f2a278745aeace757c233e75b25 (table rebalancer_extra_table_6 [id=ba79b61566954faab0f2bc52c0c0ba84]) successfully deleted
I20240502 07:59:12.650395 22229 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:12.656314 30940 tablet.cc:1722] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:12.656752 30950 tablet.cc:1722] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:12.657402 30725 tablet.cc:1722] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:12.662202 22230 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=264131b83d524000b0c7328f18e5a6e4] alter complete (version 3)
I20240502 07:59:12.725257 22613 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:12.730396 25063 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:12.735831 25193 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:12.737898 22745 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:12.743032 24800 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:12.758426 22351 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:12.765111 22230 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:12.765291 22230 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:12.767706 24932 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:12.769842 22230 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 5b189f1a92db4c3aaaed36f636115949
I20240502 07:59:12.770529 24800 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b189f1a92db4c3aaaed36f636115949 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:12 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:59:12.770546 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b189f1a92db4c3aaaed36f636115949 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:12 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:12.770900 22483 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b189f1a92db4c3aaaed36f636115949 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:12 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:12.770980 22482 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:12.772010 31035 tablet_replica.cc:330] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:59:12.772187 31035 raft_consensus.cc:2229] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:12.772349 31035 raft_consensus.cc:2258] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:12.772971 31035 ts_tablet_manager.cc:1875] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:12.775511 31037 tablet_replica.cc:330] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:12.775542 31035 ts_tablet_manager.cc:1888] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:12.775689 31037 raft_consensus.cc:2229] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:12.775916 31037 raft_consensus.cc:2258] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:12.775687 31035 log.cc:1196] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/5b189f1a92db4c3aaaed36f636115949
I20240502 07:59:12.776428 31035 ts_tablet_manager.cc:1909] T 5b189f1a92db4c3aaaed36f636115949 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:59:12.776508 31037 ts_tablet_manager.cc:1875] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:12.777182 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet 5b189f1a92db4c3aaaed36f636115949 (table rebalancer_dynamic_table_new_name [id=264131b83d524000b0c7328f18e5a6e4]) successfully deleted
I20240502 07:59:12.779145 31037 ts_tablet_manager.cc:1888] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:12.779258 31037 log.cc:1196] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/5b189f1a92db4c3aaaed36f636115949
I20240502 07:59:12.779592 31037 ts_tablet_manager.cc:1909] T 5b189f1a92db4c3aaaed36f636115949 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:59:12.785441 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 5b189f1a92db4c3aaaed36f636115949 (table rebalancer_dynamic_table_new_name [id=264131b83d524000b0c7328f18e5a6e4]) successfully deleted
I20240502 07:59:12.786842 31036 tablet_replica.cc:330] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:12.787111 31036 raft_consensus.cc:2229] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:12.787344 31036 raft_consensus.cc:2258] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:12.788120 31036 ts_tablet_manager.cc:1875] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:12.792246 31036 ts_tablet_manager.cc:1888] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:12.792541 31036 log.cc:1196] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/5b189f1a92db4c3aaaed36f636115949
I20240502 07:59:12.793557 31036 ts_tablet_manager.cc:1909] T 5b189f1a92db4c3aaaed36f636115949 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:59:12.796216 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 5b189f1a92db4c3aaaed36f636115949 (table rebalancer_dynamic_table_new_name [id=264131b83d524000b0c7328f18e5a6e4]) successfully deleted
I20240502 07:59:12.834870 22763 consensus_queue.cc:237] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:12.836158 25083 raft_consensus.cc:1261] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 07:59:12.836266 25215 raft_consensus.cc:1261] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432 [term 2 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 07:59:12.836627 30966 consensus_queue.cc:1035] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:12.836818 30598 consensus_queue.cc:1035] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:12.838662 30598 raft_consensus.cc:2932] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:12.838987 25215 raft_consensus.cc:2932] T c651077242584f57a165073de3e4a088 P 43930dee0cd5498fa8e357a34dd71432 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:12.840540 25083 raft_consensus.cc:2932] T c651077242584f57a165073de3e4a088 P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:12.841998 22230 catalog_manager.cc:5484] T c651077242584f57a165073de3e4a088 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index 2 to 4, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New cstate: current_term: 2 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:12.845590 24932 tablet_service.cc:1514] Processing DeleteTablet for tablet c651077242584f57a165073de3e4a088 with delete_type TABLET_DATA_TOMBSTONED (TS 1aecef81733c47b3b468504934357231 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:12.845850 30977 tablet_replica.cc:330] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:12.846009 30977 raft_consensus.cc:2229] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:12.846108 30977 raft_consensus.cc:2258] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:12.846524 30977 ts_tablet_manager.cc:1875] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:12.849490 30977 ts_tablet_manager.cc:1888] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20240502 07:59:12.849563 30977 log.cc:1196] T c651077242584f57a165073de3e4a088 P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/c651077242584f57a165073de3e4a088
I20240502 07:59:12.850229 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet c651077242584f57a165073de3e4a088 (table rebalancer_extra_table_2 [id=5b6a061be2624cb1852eb43e3a4cb66b]) successfully deleted
I20240502 07:59:12.862274 22500 consensus_queue.cc:237] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.863843 22634 raft_consensus.cc:1261] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:12.863915 22763 raft_consensus.cc:1261] T a0430566f5ae4ffabcf15bb8b8af6bb5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:12.864251 31073 consensus_queue.cc:1035] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:12.864490 22438 consensus_peers.cc:485] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0430566f5ae4ffabcf15bb8b8af6bb5. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:12.864580 30135 consensus_queue.cc:1035] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:12.866390 30135 raft_consensus.cc:2932] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:12.866816 22764 raft_consensus.cc:2932] T a0430566f5ae4ffabcf15bb8b8af6bb5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:12.869215 22634 raft_consensus.cc:2932] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:12.869462 22228 catalog_manager.cc:5484] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c reported cstate change: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:12.871980 22222 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:12.878347 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 1c6f0167fe5c4f9686e47e638cb25029 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d3044d3af6dc4f318d98c107324a4df6]), partition=
I20240502 07:59:12.878631 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c6f0167fe5c4f9686e47e638cb25029. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:12.880756 30927 tablet_bootstrap.cc:492] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:12.880980 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 1c6f0167fe5c4f9686e47e638cb25029 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d3044d3af6dc4f318d98c107324a4df6]), partition=
I20240502 07:59:12.881197 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c6f0167fe5c4f9686e47e638cb25029. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:12.881870 30927 tablet_bootstrap.cc:654] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:12.882977 30927 tablet_bootstrap.cc:492] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:12.883065 30927 ts_tablet_manager.cc:1367] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:59:12.883533 30927 raft_consensus.cc:348] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.883584 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 1c6f0167fe5c4f9686e47e638cb25029 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d3044d3af6dc4f318d98c107324a4df6]), partition=
I20240502 07:59:12.883654 30927 raft_consensus.cc:374] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:12.883692 30927 raft_consensus.cc:729] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:12.883818 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c6f0167fe5c4f9686e47e638cb25029. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:12.883801 30927 consensus_queue.cc:260] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.884372 30927 ts_tablet_manager.cc:1398] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:12.885154 30856 tablet_bootstrap.cc:492] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:12.887073 30856 tablet_bootstrap.cc:654] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:12.888828 31089 tablet_bootstrap.cc:492] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:12.889699 30856 tablet_bootstrap.cc:492] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:12.889787 31089 tablet_bootstrap.cc:654] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:12.889786 30856 ts_tablet_manager.cc:1367] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:12.890184 30856 raft_consensus.cc:348] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.890276 30856 raft_consensus.cc:374] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:12.890311 30856 raft_consensus.cc:729] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:12.890444 30856 consensus_queue.cc:260] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.890697 30856 ts_tablet_manager.cc:1398] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:12.890769 31089 tablet_bootstrap.cc:492] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:12.890905 31089 ts_tablet_manager.cc:1367] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:12.891342 31089 raft_consensus.cc:348] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.891503 31089 raft_consensus.cc:374] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:12.891579 31089 raft_consensus.cc:729] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:12.891722 31089 consensus_queue.cc:260] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.892103 31089 ts_tablet_manager.cc:1398] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:12.952764 30673 consensus_queue.cc:1035] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:12.959909 30954 consensus_queue.cc:1035] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:12.966775 31056 consensus_queue.cc:1035] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:12.976492 30966 consensus_queue.cc:1035] T 62c4b29fb02246c69d62765889b533ca P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:12.976794 31056 raft_consensus.cc:482] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:12.976886 31056 raft_consensus.cc:504] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.977311 31056 leader_election.cc:288] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:12.977700 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c6f0167fe5c4f9686e47e638cb25029" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:12.977834 24821 raft_consensus.cc:2454] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:12.978199 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c6f0167fe5c4f9686e47e638cb25029" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:12.978327 25083 raft_consensus.cc:2454] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:12.978600 22698 leader_election.cc:302] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:12.978775 31056 raft_consensus.cc:2781] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:12.978821 31056 raft_consensus.cc:482] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:12.978857 31056 raft_consensus.cc:3037] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:12.980019 31056 raft_consensus.cc:504] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.980386 31056 leader_election.cc:288] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:12.980803 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c6f0167fe5c4f9686e47e638cb25029" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:12.980922 24821 raft_consensus.cc:3037] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:12.981986 24821 raft_consensus.cc:2454] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:12.982411 22700 leader_election.cc:302] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:12.982407 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c6f0167fe5c4f9686e47e638cb25029" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:12.982519 25083 raft_consensus.cc:3037] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:12.983469 31056 raft_consensus.cc:2781] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:12.983500 25083 raft_consensus.cc:2454] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:12.983563 31056 raft_consensus.cc:686] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:59:12.983700 31056 consensus_queue.cc:237] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:12.987007 22225 catalog_manager.cc:5484] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:13.010432 24932 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:13.010653 25193 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:13.011710 24800 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:13.013027 22482 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:13.022445 22745 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:13.024289 22613 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:13.041244 25063 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:13.042733 22351 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:13.093235 22225 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:13.097469 24821 raft_consensus.cc:1261] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:13.097594 25083 raft_consensus.cc:1261] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:13.098075 31056 consensus_queue.cc:1035] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:13.098385 31056 consensus_queue.cc:1035] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:13.100920 31127 mvcc.cc:205] Tried to move back new op lower bound from 7023152140683157504 to 7023152140222652416. Current Snapshot: MvccSnapshot[applied={T|T < 7023152140683157504}]
I20240502 07:59:13.101676 31151 tablet.cc:1722] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:13.101999 31151 tablet.cc:1620] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:13.101281 22225 catalog_manager.cc:5576] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733) does not have the latest schema for tablet 1c6f0167fe5c4f9686e47e638cb25029 (table rebalancer_dynamic_table [id=d3044d3af6dc4f318d98c107324a4df6]). Expected version 1 got 0
I20240502 07:59:13.103132 22221 catalog_manager.cc:5576] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661) does not have the latest schema for tablet 1c6f0167fe5c4f9686e47e638cb25029 (table rebalancer_dynamic_table [id=d3044d3af6dc4f318d98c107324a4df6]). Expected version 1 got 0
I20240502 07:59:13.105463 31150 tablet.cc:1722] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:13.105814 31150 tablet.cc:1620] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:59:13.110531 22226 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=d3044d3af6dc4f318d98c107324a4df6] alter complete (version 1)
I20240502 07:59:13.110921 30725 tablet.cc:1722] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:13.111265 30725 tablet.cc:1620] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:13.112201 31056 consensus_queue.cc:1035] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:13.118865 22221 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_42"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:13.123750 30966 consensus_queue.cc:1035] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:13.127523 24800 tablet_service.cc:1467] Processing CreateTablet for tablet caf2dd9c96b94566b216692a9370f16e (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=4b6178b3995444e48f280888444ffd58]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:13.127849 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caf2dd9c96b94566b216692a9370f16e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:13.130522 30856 tablet_bootstrap.cc:492] T caf2dd9c96b94566b216692a9370f16e P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:13.131523 30856 tablet_bootstrap.cc:654] T caf2dd9c96b94566b216692a9370f16e P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:13.131934 22613 tablet_service.cc:1467] Processing CreateTablet for tablet caf2dd9c96b94566b216692a9370f16e (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=4b6178b3995444e48f280888444ffd58]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:13.132201 22614 tablet_service.cc:1467] Processing CreateTablet for tablet 5ca68d8eec874332b74df02eb55e0ff9 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=4b6178b3995444e48f280888444ffd58]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:13.132484 22351 tablet_service.cc:1467] Processing CreateTablet for tablet 6aee8f101c5e46df994dd15b49164a2b (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=4b6178b3995444e48f280888444ffd58]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:13.132536 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 6aee8f101c5e46df994dd15b49164a2b (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=4b6178b3995444e48f280888444ffd58]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:13.132679 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 5ca68d8eec874332b74df02eb55e0ff9 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=4b6178b3995444e48f280888444ffd58]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:13.132764 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aee8f101c5e46df994dd15b49164a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:13.132742 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aee8f101c5e46df994dd15b49164a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:13.132920 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ca68d8eec874332b74df02eb55e0ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:13.133898 24799 tablet_service.cc:1467] Processing CreateTablet for tablet 5ca68d8eec874332b74df02eb55e0ff9 (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=4b6178b3995444e48f280888444ffd58]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:13.134138 24799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ca68d8eec874332b74df02eb55e0ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:13.132210 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caf2dd9c96b94566b216692a9370f16e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:13.135617 24932 tablet_service.cc:1467] Processing CreateTablet for tablet caf2dd9c96b94566b216692a9370f16e (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=4b6178b3995444e48f280888444ffd58]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:13.135879 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caf2dd9c96b94566b216692a9370f16e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:13.136860 30927 tablet_bootstrap.cc:492] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:13.137612 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 6aee8f101c5e46df994dd15b49164a2b (DEFAULT_TABLE table=rebalancer_extra_table_42 [id=4b6178b3995444e48f280888444ffd58]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:13.137845 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aee8f101c5e46df994dd15b49164a2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:13.137888 30927 tablet_bootstrap.cc:654] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:13.138195 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ca68d8eec874332b74df02eb55e0ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:13.138764 31163 tablet_bootstrap.cc:492] T caf2dd9c96b94566b216692a9370f16e P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:13.139710 31163 tablet_bootstrap.cc:654] T caf2dd9c96b94566b216692a9370f16e P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:13.140390 30927 tablet_bootstrap.cc:492] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:13.140398 30856 tablet_bootstrap.cc:492] T caf2dd9c96b94566b216692a9370f16e P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:13.140478 30856 ts_tablet_manager.cc:1367] T caf2dd9c96b94566b216692a9370f16e P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20240502 07:59:13.140477 30927 ts_tablet_manager.cc:1367] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:13.140898 30856 raft_consensus.cc:348] T caf2dd9c96b94566b216692a9370f16e P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:13.140898 30927 raft_consensus.cc:348] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:13.140997 30856 raft_consensus.cc:374] T caf2dd9c96b94566b216692a9370f16e P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:13.140997 30927 raft_consensus.cc:374] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:13.141031 30927 raft_consensus.cc:729] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:13.141031 30856 raft_consensus.cc:729] T caf2dd9c96b94566b216692a9370f16e P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:13.141155 30856 consensus_queue.cc:260] T caf2dd9c96b94566b216692a9370f16e P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:13.141155 30927 consensus_queue.cc:260] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:13.141456 30856 ts_tablet_manager.cc:1398] T caf2dd9c96b94566b216692a9370f16e P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:13.141449 30927 ts_tablet_manager.cc:1398] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:13.141618 30856 tablet_bootstrap.cc:492] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:13.142628 30856 tablet_bootstrap.cc:654] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:13.143486 31164 tablet_bootstrap.cc:492] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:13.144562 31164 tablet_bootstrap.cc:654] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:13.144771 30856 tablet_bootstrap.cc:492] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:13.144858 30856 ts_tablet_manager.cc:1367] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:13.145422 30856 raft_consensus.cc:348] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:13.145608 30856 raft_consensus.cc:374] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:13.145658 30856 raft_consensus.cc:729] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:13.145807 30856 consensus_queue.cc:260] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:13.145938 31163 tablet_bootstrap.cc:492] T caf2dd9c96b94566b216692a9370f16e P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:13.146028 31163 ts_tablet_manager.cc:1367] T caf2dd9c96b94566b216692a9370f16e P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:13.146176 30856 ts_tablet_manager.cc:1398] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:13.146370 30856 tablet_bootstrap.cc:492] T 6aee8f101c5e46df994dd15b49164a2b P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:13.146534 31163 raft_consensus.cc:348] T caf2dd9c96b94566b216692a9370f16e P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:13.146737 31163 raft_consensus.cc:374] T caf2dd9c96b94566b216692a9370f16e P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:13.146817 31163 raft_consensus.cc:729] T caf2dd9c96b94566b216692a9370f16e P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:13.147007 31163 consensus_queue.cc:260] T caf2dd9c96b94566b216692a9370f16e P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:13.148028 30856 tablet_bootstrap.cc:654] T 6aee8f101c5e46df994dd15b49164a2b P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:13.148169 31163 ts_tablet_manager.cc:1398] T caf2dd9c96b94566b216692a9370f16e P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:13.148645 31164 tablet_bootstrap.cc:492] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:13.148983 31163 tablet_bootstrap.cc:492] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:13.149132 30856 tablet_bootstrap.cc:492] T 6aee8f101c5e46df994dd15b49164a2b P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:13.149205 30856 ts_tablet_manager.cc:1367] T 6aee8f101c5e46df994dd15b49164a2b P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:13.149650 30856 raft_consensus.cc:348] T 6aee8f101c5e46df994dd15b49164a2b P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:13.149737 30856 raft_consensus.cc:374] T 6aee8f101c5e46df994dd15b49164a2b P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:13.149767 30856 raft_consensus.cc:729] T 6aee8f101c5e46df994dd15b49164a2b P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:13.149883 30856 consensus_queue.cc:260] T 6aee8f101c5e46df994dd15b49164a2b P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:13.150056 31164 ts_tablet_manager.cc:1367] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:13.150146 30856 ts_tablet_manager.cc:1398] T 6aee8f101c5e46df994dd15b49164a2b P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:13.151335 31164 raft_consensus.cc:348] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:13.151557 31163 tablet_bootstrap.cc:654] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:13.152110 31164 raft_consensus.cc:374] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:13.152223 31164 raft_consensus.cc:729] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:13.152534 31164 consensus_queue.cc:260] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:13.152874 31164 ts_tablet_manager.cc:1398] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:13.153301 31163 tablet_bootstrap.cc:492] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:13.153537 31163 ts_tablet_manager.cc:1367] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:13.154008 31163 raft_consensus.cc:348] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:13.154111 31163 raft_consensus.cc:374] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:13.154147 31163 raft_consensus.cc:729] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:13.154305 31163 consensus_queue.cc:260] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:13.154558 31163 ts_tablet_manager.cc:1398] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:13.157537 31162 tablet_bootstrap.cc:492] T 6aee8f101c5e46df994dd15b49164a2b P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:13.158488 31162 tablet_bootstrap.cc:654] T 6aee8f101c5e46df994dd15b49164a2b P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:13.159744 31162 tablet_bootstrap.cc:492] T 6aee8f101c5e46df994dd15b49164a2b P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:13.159821 31162 ts_tablet_manager.cc:1367] T 6aee8f101c5e46df994dd15b49164a2b P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:13.160184 31162 raft_consensus.cc:348] T 6aee8f101c5e46df994dd15b49164a2b P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:13.160277 31162 raft_consensus.cc:374] T 6aee8f101c5e46df994dd15b49164a2b P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:13.160300 31162 raft_consensus.cc:729] T 6aee8f101c5e46df994dd15b49164a2b P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:13.160485 31162 consensus_queue.cc:260] T 6aee8f101c5e46df994dd15b49164a2b P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:13.160769 31162 ts_tablet_manager.cc:1398] T 6aee8f101c5e46df994dd15b49164a2b P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:13.161091 31162 tablet_bootstrap.cc:492] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:13.162225 31162 tablet_bootstrap.cc:654] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:13.164049 31162 tablet_bootstrap.cc:492] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:13.164150 31162 ts_tablet_manager.cc:1367] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:13.164680 31162 raft_consensus.cc:348] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:13.164777 31162 raft_consensus.cc:374] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:13.164800 31162 raft_consensus.cc:729] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:13.164920 31162 consensus_queue.cc:260] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:13.165179 31162 ts_tablet_manager.cc:1398] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:13.215090 22222 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:13.223553 31150 tablet.cc:1722] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:13.223915 31150 tablet.cc:1620] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:59:13.225606 31151 tablet.cc:1722] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:13.225908 31151 tablet.cc:1620] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:13.226166 30725 tablet.cc:1722] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:13.226449 30725 tablet.cc:1620] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b: MemRowSet was empty: no flush needed.
I20240502 07:59:13.232751 22222 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=d3044d3af6dc4f318d98c107324a4df6] alter complete (version 2)
I20240502 07:59:13.243908 31161 raft_consensus.cc:482] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:13.244031 31161 raft_consensus.cc:504] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:13.244469 31161 leader_election.cc:288] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:13.244939 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ca68d8eec874332b74df02eb55e0ff9" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:13.244956 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ca68d8eec874332b74df02eb55e0ff9" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:13.245079 22634 raft_consensus.cc:2454] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:13.245079 22370 raft_consensus.cc:2454] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:13.245376 24757 leader_election.cc:302] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:13.245539 31161 raft_consensus.cc:2781] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:13.245591 31161 raft_consensus.cc:482] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:13.245635 31161 raft_consensus.cc:3037] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:13.246618 31161 raft_consensus.cc:504] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:13.246951 31161 leader_election.cc:288] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:13.247381 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ca68d8eec874332b74df02eb55e0ff9" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:13.247383 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ca68d8eec874332b74df02eb55e0ff9" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:13.247493 22634 raft_consensus.cc:3037] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:13.247489 22370 raft_consensus.cc:3037] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:13.248307 22370 raft_consensus.cc:2454] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:13.248519 22634 raft_consensus.cc:2454] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:13.248561 24757 leader_election.cc:302] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:13.248709 29297 raft_consensus.cc:2781] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:13.248804 29297 raft_consensus.cc:686] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:59:13.248955 29297 consensus_queue.cc:237] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:13.251597 22228 catalog_manager.cc:5484] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:13.259256 30966 raft_consensus.cc:482] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:13.259383 30966 raft_consensus.cc:504] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:13.259783 30966 leader_election.cc:288] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:13.260044 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aee8f101c5e46df994dd15b49164a2b" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:13.260185 24821 raft_consensus.cc:2454] T 6aee8f101c5e46df994dd15b49164a2b P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:13.260177 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aee8f101c5e46df994dd15b49164a2b" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:13.260314 22370 raft_consensus.cc:2454] T 6aee8f101c5e46df994dd15b49164a2b P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:13.260553 22700 leader_election.cc:302] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:13.260841 30966 raft_consensus.cc:2781] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:13.260900 30966 raft_consensus.cc:482] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:13.260962 30966 raft_consensus.cc:3037] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:13.261842 30966 raft_consensus.cc:504] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:13.262180 30966 leader_election.cc:288] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:13.262506 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aee8f101c5e46df994dd15b49164a2b" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:13.262532 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aee8f101c5e46df994dd15b49164a2b" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:13.262624 22370 raft_consensus.cc:3037] T 6aee8f101c5e46df994dd15b49164a2b P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:13.262625 24821 raft_consensus.cc:3037] T 6aee8f101c5e46df994dd15b49164a2b P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:13.263363 22370 raft_consensus.cc:2454] T 6aee8f101c5e46df994dd15b49164a2b P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:13.263551 24821 raft_consensus.cc:2454] T 6aee8f101c5e46df994dd15b49164a2b P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:13.263648 22701 leader_election.cc:302] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:13.263816 30966 raft_consensus.cc:2781] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:13.263906 30966 raft_consensus.cc:686] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:59:13.264071 30966 consensus_queue.cc:237] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:13.266669 22228 catalog_manager.cc:5484] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:13.337508 22228 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:13.343055 31150 tablet.cc:1722] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:13.343492 30725 tablet.cc:1722] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:13.344193 31151 tablet.cc:1722] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:13.350941 22228 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=d3044d3af6dc4f318d98c107324a4df6] alter complete (version 3)
I20240502 07:59:13.361274 31206 ts_tablet_manager.cc:902] T a0430566f5ae4ffabcf15bb8b8af6bb5 P 4b4e7eeb7f3f42119c0363a79febc54b: Initiating tablet copy from peer fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:13.361812 31206 tablet_copy_client.cc:308] T a0430566f5ae4ffabcf15bb8b8af6bb5 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.130:39625
I20240502 07:59:13.362288 22523 tablet_copy_service.cc:140] P fabddb16c85944648a710ec52fc7244c: Received BeginTabletCopySession request for tablet a0430566f5ae4ffabcf15bb8b8af6bb5 from peer 4b4e7eeb7f3f42119c0363a79febc54b ({username='slave'} at 127.15.129.133:49719)
I20240502 07:59:13.362386 22523 tablet_copy_service.cc:161] P fabddb16c85944648a710ec52fc7244c: Beginning new tablet copy session on tablet a0430566f5ae4ffabcf15bb8b8af6bb5 from peer 4b4e7eeb7f3f42119c0363a79febc54b at {username='slave'} at 127.15.129.133:49719: session id = 4b4e7eeb7f3f42119c0363a79febc54b-a0430566f5ae4ffabcf15bb8b8af6bb5
I20240502 07:59:13.363168 22523 tablet_copy_source_session.cc:205] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:13.363988 31206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0430566f5ae4ffabcf15bb8b8af6bb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:13.366060 31206 tablet_copy_client.cc:783] T a0430566f5ae4ffabcf15bb8b8af6bb5 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:13.366288 31206 tablet_copy_client.cc:647] T a0430566f5ae4ffabcf15bb8b8af6bb5 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:13.367420 31206 tablet_copy_client.cc:523] T a0430566f5ae4ffabcf15bb8b8af6bb5 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:13.368710 31206 tablet_bootstrap.cc:492] T a0430566f5ae4ffabcf15bb8b8af6bb5 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:13.403641 31206 tablet_bootstrap.cc:492] T a0430566f5ae4ffabcf15bb8b8af6bb5 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:13.403903 31206 tablet_bootstrap.cc:492] T a0430566f5ae4ffabcf15bb8b8af6bb5 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap complete.
I20240502 07:59:13.404044 31206 ts_tablet_manager.cc:1367] T a0430566f5ae4ffabcf15bb8b8af6bb5 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.035s user 0.019s sys 0.000s
I20240502 07:59:13.404425 31206 raft_consensus.cc:348] T a0430566f5ae4ffabcf15bb8b8af6bb5 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:13.404533 31206 raft_consensus.cc:729] T a0430566f5ae4ffabcf15bb8b8af6bb5 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:13.404634 31206 consensus_queue.cc:260] T a0430566f5ae4ffabcf15bb8b8af6bb5 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:13.404846 31206 ts_tablet_manager.cc:1398] T a0430566f5ae4ffabcf15bb8b8af6bb5 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:13.405417 22523 tablet_copy_service.cc:342] P fabddb16c85944648a710ec52fc7244c: Request end of tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-a0430566f5ae4ffabcf15bb8b8af6bb5 received from {username='slave'} at 127.15.129.133:49719
I20240502 07:59:13.405514 22523 tablet_copy_service.cc:433] P fabddb16c85944648a710ec52fc7244c: ending tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-a0430566f5ae4ffabcf15bb8b8af6bb5 on tablet a0430566f5ae4ffabcf15bb8b8af6bb5 with peer 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:13.454625 22222 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:13.454811 22222 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:13.456833 22222 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 1c6f0167fe5c4f9686e47e638cb25029
I20240502 07:59:13.457422 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c6f0167fe5c4f9686e47e638cb25029 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:13 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:13.457451 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c6f0167fe5c4f9686e47e638cb25029 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:13 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:13.457537 24800 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c6f0167fe5c4f9686e47e638cb25029 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:13 UTC) from {username='slave'} at 127.0.0.1:36878
I20240502 07:59:13.457880 31214 tablet_replica.cc:330] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:59:13.458034 31214 raft_consensus.cc:2229] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:13.458153 31214 raft_consensus.cc:2258] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:13.458734 31214 ts_tablet_manager.cc:1875] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:13.459007 31213 tablet_replica.cc:330] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:13.459204 31213 raft_consensus.cc:2229] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:13.459506 31213 raft_consensus.cc:2258] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:13.460028 31213 ts_tablet_manager.cc:1875] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:13.461205 31214 ts_tablet_manager.cc:1888] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:13.461273 31214 log.cc:1196] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/1c6f0167fe5c4f9686e47e638cb25029
I20240502 07:59:13.461575 31214 ts_tablet_manager.cc:1909] T 1c6f0167fe5c4f9686e47e638cb25029 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting consensus metadata
I20240502 07:59:13.462213 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet 1c6f0167fe5c4f9686e47e638cb25029 (table rebalancer_dynamic_table_new_name [id=d3044d3af6dc4f318d98c107324a4df6]) successfully deleted
I20240502 07:59:13.462250 31213 ts_tablet_manager.cc:1888] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:13.462306 31213 log.cc:1196] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/1c6f0167fe5c4f9686e47e638cb25029
I20240502 07:59:13.462590 31213 ts_tablet_manager.cc:1909] T 1c6f0167fe5c4f9686e47e638cb25029 P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:59:13.463240 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet 1c6f0167fe5c4f9686e47e638cb25029 (table rebalancer_dynamic_table_new_name [id=d3044d3af6dc4f318d98c107324a4df6]) successfully deleted
I20240502 07:59:13.467950 31212 tablet_replica.cc:330] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:13.468101 31212 raft_consensus.cc:2229] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:13.468304 31212 raft_consensus.cc:2258] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:13.468835 31212 ts_tablet_manager.cc:1875] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:13.471096 31212 ts_tablet_manager.cc:1888] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:13.471179 31212 log.cc:1196] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/1c6f0167fe5c4f9686e47e638cb25029
I20240502 07:59:13.471521 31212 ts_tablet_manager.cc:1909] T 1c6f0167fe5c4f9686e47e638cb25029 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:59:13.472137 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 1c6f0167fe5c4f9686e47e638cb25029 (table rebalancer_dynamic_table_new_name [id=d3044d3af6dc4f318d98c107324a4df6]) successfully deleted
I20240502 07:59:13.473788 31130 raft_consensus.cc:482] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:13.473875 31130 raft_consensus.cc:504] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:13.474340 31130 leader_election.cc:288] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:13.474640 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caf2dd9c96b94566b216692a9370f16e" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:13.474716 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caf2dd9c96b94566b216692a9370f16e" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:13.474776 24821 raft_consensus.cc:2454] T caf2dd9c96b94566b216692a9370f16e P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:13.474825 22634 raft_consensus.cc:2454] T caf2dd9c96b94566b216692a9370f16e P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:13.475025 24887 leader_election.cc:302] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:13.475208 31130 raft_consensus.cc:2781] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:13.475262 31130 raft_consensus.cc:482] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:13.475313 31130 raft_consensus.cc:3037] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:13.476063 31130 raft_consensus.cc:504] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:13.476377 31130 leader_election.cc:288] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:13.476651 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caf2dd9c96b94566b216692a9370f16e" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:13.476758 22634 raft_consensus.cc:3037] T caf2dd9c96b94566b216692a9370f16e P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:13.477428 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caf2dd9c96b94566b216692a9370f16e" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:13.477527 24821 raft_consensus.cc:3037] T caf2dd9c96b94566b216692a9370f16e P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:13.477576 22634 raft_consensus.cc:2454] T caf2dd9c96b94566b216692a9370f16e P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:13.477839 24888 leader_election.cc:302] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:13.477993 31130 raft_consensus.cc:2781] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:13.478168 31130 raft_consensus.cc:686] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:13.478310 31130 consensus_queue.cc:237] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:13.478506 24821 raft_consensus.cc:2454] T caf2dd9c96b94566b216692a9370f16e P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:13.480309 22222 catalog_manager.cc:5484] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:13.509308 25063 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:13.509346 24932 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:13.509716 24800 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:13.515290 25193 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:13.521481 22614 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:13.529510 22482 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:13.534657 22745 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:13.557861 22352 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:13.558828 22222 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:13.566624 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 9e053de314dd4a6bb109f77f7f3c2b5a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d00d64e385d249c9890fa7e6e3b83dbf]), partition=
I20240502 07:59:13.566689 22614 tablet_service.cc:1467] Processing CreateTablet for tablet 9e053de314dd4a6bb109f77f7f3c2b5a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d00d64e385d249c9890fa7e6e3b83dbf]), partition=
I20240502 07:59:13.566918 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e053de314dd4a6bb109f77f7f3c2b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:13.566931 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e053de314dd4a6bb109f77f7f3c2b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:13.569454 31163 tablet_bootstrap.cc:492] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:13.569561 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 9e053de314dd4a6bb109f77f7f3c2b5a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d00d64e385d249c9890fa7e6e3b83dbf]), partition=
I20240502 07:59:13.569793 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e053de314dd4a6bb109f77f7f3c2b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:13.570416 31163 tablet_bootstrap.cc:654] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:13.570955 31164 tablet_bootstrap.cc:492] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:13.571843 31164 tablet_bootstrap.cc:654] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:13.572467 30927 tablet_bootstrap.cc:492] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:13.573273 30927 tablet_bootstrap.cc:654] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:13.577005 31163 tablet_bootstrap.cc:492] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:13.577116 31163 ts_tablet_manager.cc:1367] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20240502 07:59:13.578359 31163 raft_consensus.cc:348] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:13.578500 31163 raft_consensus.cc:374] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:13.578554 31163 raft_consensus.cc:729] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:13.578665 31163 consensus_queue.cc:260] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:13.579699 31163 ts_tablet_manager.cc:1398] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:13.587905 30927 tablet_bootstrap.cc:492] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:13.588004 30927 ts_tablet_manager.cc:1367] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 07:59:13.588482 30927 raft_consensus.cc:348] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:13.588586 30927 raft_consensus.cc:374] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:13.588610 30927 raft_consensus.cc:729] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:13.588743 30927 consensus_queue.cc:260] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:13.588991 30927 ts_tablet_manager.cc:1398] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:13.590531 31164 tablet_bootstrap.cc:492] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:13.590637 31164 ts_tablet_manager.cc:1367] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.002s
I20240502 07:59:13.591189 31164 raft_consensus.cc:348] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:13.591306 31164 raft_consensus.cc:374] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:13.591343 31164 raft_consensus.cc:729] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:13.591464 31164 consensus_queue.cc:260] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:13.591748 31164 ts_tablet_manager.cc:1398] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:13.626127 22634 consensus_queue.cc:237] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:13.627519 24821 raft_consensus.cc:1261] T 8477b961ce384edea078851122e196e9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:13.627581 22370 raft_consensus.cc:1261] T 8477b961ce384edea078851122e196e9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:13.627918 30776 consensus_queue.cc:1035] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:13.628074 29693 consensus_queue.cc:1035] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:13.629804 29693 raft_consensus.cc:2932] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:13.630344 22370 raft_consensus.cc:2932] T 8477b961ce384edea078851122e196e9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:13.630661 24821 raft_consensus.cc:2932] T 8477b961ce384edea078851122e196e9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:13.633649 22222 catalog_manager.cc:5484] T 8477b961ce384edea078851122e196e9 P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: config changed from index 4 to 5, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:13.636745 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet 8477b961ce384edea078851122e196e9 with delete_type TABLET_DATA_TOMBSTONED (TS f00f3a1876444ad2bf9c919068ab8014 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:13.636997 31212 tablet_replica.cc:330] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:13.637164 31212 raft_consensus.cc:2229] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:13.637358 31212 raft_consensus.cc:2258] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:13.637909 31212 ts_tablet_manager.cc:1875] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:13.640121 31212 ts_tablet_manager.cc:1888] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:13.640197 31212 log.cc:1196] T 8477b961ce384edea078851122e196e9 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/8477b961ce384edea078851122e196e9
I20240502 07:59:13.640751 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 8477b961ce384edea078851122e196e9 (table rebalance_test_table_7 [id=255d8aedf8f845a1b8080105bc5b3a93]) successfully deleted
I20240502 07:59:13.669991 31056 consensus_queue.cc:1035] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:13.673729 31056 consensus_queue.cc:1035] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:13.745254 31161 consensus_queue.cc:1035] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:13.747879 31056 raft_consensus.cc:482] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:13.748004 31056 raft_consensus.cc:504] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:13.748396 31056 leader_election.cc:288] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:13.748824 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e053de314dd4a6bb109f77f7f3c2b5a" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:13.748951 24951 raft_consensus.cc:2454] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:13.748943 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e053de314dd4a6bb109f77f7f3c2b5a" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:13.749084 22632 raft_consensus.cc:2454] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:13.749760 22701 leader_election.cc:302] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:13.750437 31056 raft_consensus.cc:2781] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:13.750509 31056 raft_consensus.cc:482] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:13.750550 31056 raft_consensus.cc:3037] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:13.751731 31056 raft_consensus.cc:504] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:13.752089 31056 leader_election.cc:288] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:13.753063 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e053de314dd4a6bb109f77f7f3c2b5a" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:13.753108 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e053de314dd4a6bb109f77f7f3c2b5a" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:13.753178 24951 raft_consensus.cc:3037] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:13.753234 22632 raft_consensus.cc:3037] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:13.754184 24951 raft_consensus.cc:2454] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:13.754431 22632 raft_consensus.cc:2454] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:13.755445 22701 leader_election.cc:302] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:13.755982 30966 raft_consensus.cc:2781] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:13.756071 30966 raft_consensus.cc:686] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:59:13.756210 30966 consensus_queue.cc:237] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:13.759752 22222 catalog_manager.cc:5484] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:13.760849 31161 consensus_queue.cc:1035] T 5ca68d8eec874332b74df02eb55e0ff9 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:13.785312 24932 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:13.788827 24800 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:13.794909 25063 tablet_service.cc:1429] Tablet server has 8 leaders and 0 scanners
I20240502 07:59:13.797111 22482 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:13.797607 22745 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:13.802529 25193 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:13.831071 22614 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:13.831558 22352 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:13.880403 22632 consensus_queue.cc:237] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:13.881788 22764 raft_consensus.cc:1261] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:13.881788 25215 raft_consensus.cc:1261] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:13.882200 29693 consensus_queue.cc:1035] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:13.882665 30776 consensus_queue.cc:1035] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:13.884136 29693 raft_consensus.cc:2932] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:13.884640 25215 raft_consensus.cc:2932] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:13.888062 22764 raft_consensus.cc:2932] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:13.888329 22222 catalog_manager.cc:5484] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index 4 to 5, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) evicted. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:13.891165 22482 tablet_service.cc:1514] Processing DeleteTablet for tablet 2241a741bf85433b9fea512e9c530416 with delete_type TABLET_DATA_TOMBSTONED (TS fabddb16c85944648a710ec52fc7244c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:13.891510 31326 tablet_replica.cc:330] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:13.891669 31326 raft_consensus.cc:2229] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:13.891840 31326 raft_consensus.cc:2258] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:13.892401 31326 ts_tablet_manager.cc:1875] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:13.895169 31326 ts_tablet_manager.cc:1888] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:13.895254 31326 log.cc:1196] T 2241a741bf85433b9fea512e9c530416 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/2241a741bf85433b9fea512e9c530416
I20240502 07:59:13.895792 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 2241a741bf85433b9fea512e9c530416 (table rebalance_test_table_11 [id=423ebbc6961345159cc3d4c9668fe1dd]) successfully deleted
I20240502 07:59:13.897063 22224 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:13.901935 22632 raft_consensus.cc:1261] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:13.902329 24952 raft_consensus.cc:1261] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:13.902702 30598 consensus_queue.cc:1035] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:13.902835 31056 consensus_queue.cc:1035] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:13.905845 31339 tablet.cc:1722] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:13.906179 22222 catalog_manager.cc:5576] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703) does not have the latest schema for tablet 9e053de314dd4a6bb109f77f7f3c2b5a (table rebalancer_dynamic_table [id=d00d64e385d249c9890fa7e6e3b83dbf]). Expected version 1 got 0
I20240502 07:59:13.906297 31339 tablet.cc:1620] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:59:13.908905 22224 catalog_manager.cc:5576] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995) does not have the latest schema for tablet 9e053de314dd4a6bb109f77f7f3c2b5a (table rebalancer_dynamic_table [id=d00d64e385d249c9890fa7e6e3b83dbf]). Expected version 1 got 0
I20240502 07:59:13.909718 31340 tablet.cc:1722] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:13.910059 31340 tablet.cc:1620] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
I20240502 07:59:13.911685 22222 catalog_manager.cc:5576] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995) does not have the latest schema for tablet 9e053de314dd4a6bb109f77f7f3c2b5a (table rebalancer_dynamic_table [id=d00d64e385d249c9890fa7e6e3b83dbf]). Expected version 1 got 0
I20240502 07:59:13.913064 22229 catalog_manager.cc:5576] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703) does not have the latest schema for tablet 9e053de314dd4a6bb109f77f7f3c2b5a (table rebalancer_dynamic_table [id=d00d64e385d249c9890fa7e6e3b83dbf]). Expected version 1 got 0
I20240502 07:59:13.917575 22224 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=d00d64e385d249c9890fa7e6e3b83dbf] alter complete (version 1)
I20240502 07:59:13.918910 31342 tablet.cc:1722] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:13.919202 31342 tablet.cc:1620] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:13.923797 31130 consensus_queue.cc:1035] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:13.929208 29729 consensus_queue.cc:1035] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:13.967797 24820 raft_consensus.cc:1203] T a0430566f5ae4ffabcf15bb8b8af6bb5 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:13.985134 22372 consensus_queue.cc:237] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:13.986673 22632 raft_consensus.cc:1261] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:13.986825 22307 consensus_peers.cc:485] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f5f0690b92c140c49fe10ff46e0136df. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:13.986861 22499 raft_consensus.cc:1261] T f5f0690b92c140c49fe10ff46e0136df P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:13.987221 31386 consensus_queue.cc:1035] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:13.987397 29808 consensus_queue.cc:1035] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:13.989153 31386 raft_consensus.cc:2932] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:13.990161 22632 raft_consensus.cc:2932] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:13.990996 22499 raft_consensus.cc:2932] T f5f0690b92c140c49fe10ff46e0136df P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:13.994134 22224 catalog_manager.cc:5484] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:14.010435 22222 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_44"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:14.016644 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 69f190895a344889b24868b3a2e4a41a (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=4df0ca47886d4ad88461ccaec02245e6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:14.017002 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69f190895a344889b24868b3a2e4a41a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.016989 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 89889b6805684edeb79afd3fa3a55dea (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=4df0ca47886d4ad88461ccaec02245e6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:14.017244 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89889b6805684edeb79afd3fa3a55dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.017637 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 69f190895a344889b24868b3a2e4a41a (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=4df0ca47886d4ad88461ccaec02245e6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:14.017882 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69f190895a344889b24868b3a2e4a41a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.019368 30927 tablet_bootstrap.cc:492] T 89889b6805684edeb79afd3fa3a55dea P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:14.020092 31401 tablet_bootstrap.cc:492] T 69f190895a344889b24868b3a2e4a41a P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:14.020344 30927 tablet_bootstrap.cc:654] T 89889b6805684edeb79afd3fa3a55dea P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.020810 22483 tablet_service.cc:1467] Processing CreateTablet for tablet 5246ae54d56b419bbfef485e7bc18ab9 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=4df0ca47886d4ad88461ccaec02245e6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:14.021086 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5246ae54d56b419bbfef485e7bc18ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.021185 22482 tablet_service.cc:1467] Processing CreateTablet for tablet 89889b6805684edeb79afd3fa3a55dea (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=4df0ca47886d4ad88461ccaec02245e6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:14.021484 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89889b6805684edeb79afd3fa3a55dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.021540 31401 tablet_bootstrap.cc:654] T 69f190895a344889b24868b3a2e4a41a P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.022759 31401 tablet_bootstrap.cc:492] T 69f190895a344889b24868b3a2e4a41a P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:14.022838 31401 ts_tablet_manager.cc:1367] T 69f190895a344889b24868b3a2e4a41a P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:14.023041 31404 tablet_bootstrap.cc:492] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:14.023308 31401 raft_consensus.cc:348] T 69f190895a344889b24868b3a2e4a41a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:14.023451 31401 raft_consensus.cc:374] T 69f190895a344889b24868b3a2e4a41a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.023514 31401 raft_consensus.cc:729] T 69f190895a344889b24868b3a2e4a41a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.023667 31401 consensus_queue.cc:260] T 69f190895a344889b24868b3a2e4a41a P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:14.024006 31401 ts_tablet_manager.cc:1398] T 69f190895a344889b24868b3a2e4a41a P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:14.024222 31404 tablet_bootstrap.cc:654] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.024619 25193 tablet_service.cc:1467] Processing CreateTablet for tablet 89889b6805684edeb79afd3fa3a55dea (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=4df0ca47886d4ad88461ccaec02245e6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:14.024634 22614 tablet_service.cc:1467] Processing CreateTablet for tablet 5246ae54d56b419bbfef485e7bc18ab9 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=4df0ca47886d4ad88461ccaec02245e6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:14.024891 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89889b6805684edeb79afd3fa3a55dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.024976 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5246ae54d56b419bbfef485e7bc18ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.025816 24932 tablet_service.cc:1467] Processing CreateTablet for tablet 69f190895a344889b24868b3a2e4a41a (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=4df0ca47886d4ad88461ccaec02245e6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:14.026087 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69f190895a344889b24868b3a2e4a41a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.027230 22222 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:14.028510 31164 tablet_bootstrap.cc:492] T 69f190895a344889b24868b3a2e4a41a P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:14.028772 31405 tablet_bootstrap.cc:492] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:14.029559 31164 tablet_bootstrap.cc:654] T 69f190895a344889b24868b3a2e4a41a P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.030056 30927 tablet_bootstrap.cc:492] T 89889b6805684edeb79afd3fa3a55dea P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:14.030025 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 5246ae54d56b419bbfef485e7bc18ab9 (DEFAULT_TABLE table=rebalancer_extra_table_44 [id=4df0ca47886d4ad88461ccaec02245e6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:14.030164 30927 ts_tablet_manager.cc:1367] T 89889b6805684edeb79afd3fa3a55dea P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 07:59:14.030304 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5246ae54d56b419bbfef485e7bc18ab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.030659 30927 raft_consensus.cc:348] T 89889b6805684edeb79afd3fa3a55dea P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:14.030778 30927 raft_consensus.cc:374] T 89889b6805684edeb79afd3fa3a55dea P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.030808 30927 raft_consensus.cc:729] T 89889b6805684edeb79afd3fa3a55dea P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.030921 30927 consensus_queue.cc:260] T 89889b6805684edeb79afd3fa3a55dea P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:14.031170 30927 ts_tablet_manager.cc:1398] T 89889b6805684edeb79afd3fa3a55dea P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:14.031950 31164 tablet_bootstrap.cc:492] T 69f190895a344889b24868b3a2e4a41a P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:14.032027 31164 ts_tablet_manager.cc:1367] T 69f190895a344889b24868b3a2e4a41a P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 07:59:14.032617 31405 tablet_bootstrap.cc:654] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.032658 31164 raft_consensus.cc:348] T 69f190895a344889b24868b3a2e4a41a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:14.032794 31164 raft_consensus.cc:374] T 69f190895a344889b24868b3a2e4a41a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.032833 31164 raft_consensus.cc:729] T 69f190895a344889b24868b3a2e4a41a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.032953 31164 consensus_queue.cc:260] T 69f190895a344889b24868b3a2e4a41a P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:14.033278 31164 ts_tablet_manager.cc:1398] T 69f190895a344889b24868b3a2e4a41a P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 07:59:14.033651 31400 tablet_bootstrap.cc:492] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:14.035477 31163 tablet_bootstrap.cc:492] T 5246ae54d56b419bbfef485e7bc18ab9 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:14.036401 31163 tablet_bootstrap.cc:654] T 5246ae54d56b419bbfef485e7bc18ab9 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.037791 31400 tablet_bootstrap.cc:654] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.038820 31163 tablet_bootstrap.cc:492] T 5246ae54d56b419bbfef485e7bc18ab9 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:14.038894 31163 ts_tablet_manager.cc:1367] T 5246ae54d56b419bbfef485e7bc18ab9 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:14.039292 31163 raft_consensus.cc:348] T 5246ae54d56b419bbfef485e7bc18ab9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:14.039391 31163 raft_consensus.cc:374] T 5246ae54d56b419bbfef485e7bc18ab9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.039418 31163 raft_consensus.cc:729] T 5246ae54d56b419bbfef485e7bc18ab9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.039538 31163 consensus_queue.cc:260] T 5246ae54d56b419bbfef485e7bc18ab9 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:14.039871 31163 ts_tablet_manager.cc:1398] T 5246ae54d56b419bbfef485e7bc18ab9 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:14.041296 31405 tablet_bootstrap.cc:492] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:14.041461 31405 ts_tablet_manager.cc:1367] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 07:59:14.041944 31405 raft_consensus.cc:348] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:14.042053 31405 raft_consensus.cc:374] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.042088 31405 raft_consensus.cc:729] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.042213 31405 consensus_queue.cc:260] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:14.042649 31405 ts_tablet_manager.cc:1398] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:14.043416 31400 tablet_bootstrap.cc:492] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:14.043605 31400 ts_tablet_manager.cc:1367] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 07:59:14.044106 31400 raft_consensus.cc:348] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:14.044282 31400 raft_consensus.cc:374] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.044358 31400 raft_consensus.cc:729] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.044591 31400 consensus_queue.cc:260] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:14.045804 31400 ts_tablet_manager.cc:1398] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:14.048703 31339 tablet.cc:1722] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:14.049021 31339 tablet.cc:1620] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:59:14.049062 31340 tablet.cc:1722] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:14.049361 31340 tablet.cc:1620] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
I20240502 07:59:14.055625 31342 tablet.cc:1722] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:14.055945 31342 tablet.cc:1620] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:14.056437 31400 tablet_bootstrap.cc:492] T 5246ae54d56b419bbfef485e7bc18ab9 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:14.057098 31404 tablet_bootstrap.cc:492] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:14.057278 31404 ts_tablet_manager.cc:1367] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.000s
I20240502 07:59:14.057504 31400 tablet_bootstrap.cc:654] T 5246ae54d56b419bbfef485e7bc18ab9 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.057825 31404 raft_consensus.cc:348] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:14.058017 31404 raft_consensus.cc:374] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.058092 31404 raft_consensus.cc:729] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.058269 31404 consensus_queue.cc:260] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:14.058672 31404 ts_tablet_manager.cc:1398] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:14.061868 22222 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=d00d64e385d249c9890fa7e6e3b83dbf] alter complete (version 2)
I20240502 07:59:14.064553 30670 raft_consensus.cc:482] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:14.064692 30670 raft_consensus.cc:504] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:14.065110 30670 leader_election.cc:288] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:14.065161 31400 tablet_bootstrap.cc:492] T 5246ae54d56b419bbfef485e7bc18ab9 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:14.065245 31400 ts_tablet_manager.cc:1367] T 5246ae54d56b419bbfef485e7bc18ab9 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20240502 07:59:14.065431 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5246ae54d56b419bbfef485e7bc18ab9" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:14.065577 22632 raft_consensus.cc:2454] T 5246ae54d56b419bbfef485e7bc18ab9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:14.065693 31400 raft_consensus.cc:348] T 5246ae54d56b419bbfef485e7bc18ab9 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:14.065712 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5246ae54d56b419bbfef485e7bc18ab9" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:14.065795 31400 raft_consensus.cc:374] T 5246ae54d56b419bbfef485e7bc18ab9 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.065848 31400 raft_consensus.cc:729] T 5246ae54d56b419bbfef485e7bc18ab9 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.065871 22439 leader_election.cc:302] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:14.065973 31400 consensus_queue.cc:260] T 5246ae54d56b419bbfef485e7bc18ab9 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:14.066028 30670 raft_consensus.cc:2781] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:14.066068 30670 raft_consensus.cc:482] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:14.066099 30670 raft_consensus.cc:3037] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:14.066208 25084 raft_consensus.cc:2454] T 5246ae54d56b419bbfef485e7bc18ab9 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:14.066277 31400 ts_tablet_manager.cc:1398] T 5246ae54d56b419bbfef485e7bc18ab9 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 07:59:14.062355 31404 tablet_bootstrap.cc:492] T 89889b6805684edeb79afd3fa3a55dea P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:14.067261 30670 raft_consensus.cc:504] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:14.067605 30670 leader_election.cc:288] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
W20240502 07:59:14.067641 22681 tablet.cc:2369] T 5246ae54d56b419bbfef485e7bc18ab9 P e51b10af49484aaa8f0267aebc1bf27f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:59:14.068313 31404 tablet_bootstrap.cc:654] T 89889b6805684edeb79afd3fa3a55dea P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.069752 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5246ae54d56b419bbfef485e7bc18ab9" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:14.069856 25084 raft_consensus.cc:3037] T 5246ae54d56b419bbfef485e7bc18ab9 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:14.070811 25084 raft_consensus.cc:2454] T 5246ae54d56b419bbfef485e7bc18ab9 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:14.071328 31404 tablet_bootstrap.cc:492] T 89889b6805684edeb79afd3fa3a55dea P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:14.071321 22436 leader_election.cc:302] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:14.071465 31404 ts_tablet_manager.cc:1367] T 89889b6805684edeb79afd3fa3a55dea P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20240502 07:59:14.071650 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5246ae54d56b419bbfef485e7bc18ab9" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:14.071756 22632 raft_consensus.cc:3037] T 5246ae54d56b419bbfef485e7bc18ab9 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:14.071887 31404 raft_consensus.cc:348] T 89889b6805684edeb79afd3fa3a55dea P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:14.071983 31404 raft_consensus.cc:374] T 89889b6805684edeb79afd3fa3a55dea P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.072011 31404 raft_consensus.cc:729] T 89889b6805684edeb79afd3fa3a55dea P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.072106 31404 consensus_queue.cc:260] T 89889b6805684edeb79afd3fa3a55dea P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:14.072562 30670 raft_consensus.cc:2781] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:14.072626 30670 raft_consensus.cc:686] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:59:14.072701 22632 raft_consensus.cc:2454] T 5246ae54d56b419bbfef485e7bc18ab9 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:14.072732 30670 consensus_queue.cc:237] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:14.074143 31404 ts_tablet_manager.cc:1398] T 89889b6805684edeb79afd3fa3a55dea P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 07:59:14.075976 22224 catalog_manager.cc:5484] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
W20240502 07:59:14.119962 22550 tablet.cc:2369] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:59:14.153231 31418 raft_consensus.cc:482] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:14.153388 31418 raft_consensus.cc:504] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:14.153847 31418 leader_election.cc:288] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:14.154248 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f190895a344889b24868b3a2e4a41a" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:14.154278 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f190895a344889b24868b3a2e4a41a" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:14.154388 22372 raft_consensus.cc:2454] T 69f190895a344889b24868b3a2e4a41a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:14.154407 24952 raft_consensus.cc:2454] T 69f190895a344889b24868b3a2e4a41a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:14.154712 25020 leader_election.cc:302] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:14.154911 31418 raft_consensus.cc:2781] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:14.154970 31418 raft_consensus.cc:482] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:14.155021 31418 raft_consensus.cc:3037] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:14.156164 31418 raft_consensus.cc:504] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:14.156466 31418 leader_election.cc:288] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:14.156827 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f190895a344889b24868b3a2e4a41a" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:14.156967 22372 raft_consensus.cc:3037] T 69f190895a344889b24868b3a2e4a41a P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:14.157790 22372 raft_consensus.cc:2454] T 69f190895a344889b24868b3a2e4a41a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:14.158001 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69f190895a344889b24868b3a2e4a41a" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:14.158107 24952 raft_consensus.cc:3037] T 69f190895a344889b24868b3a2e4a41a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:14.158106 25020 leader_election.cc:302] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:14.158253 31418 raft_consensus.cc:2781] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:14.158345 31418 raft_consensus.cc:686] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:14.158471 31418 consensus_queue.cc:237] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:14.159106 24952 raft_consensus.cc:2454] T 69f190895a344889b24868b3a2e4a41a P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:14.160840 22222 catalog_manager.cc:5484] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
W20240502 07:59:14.163090 22419 tablet.cc:2369] T 69f190895a344889b24868b3a2e4a41a P b533a7d8d97943609c6e28e3e36b78a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:59:14.168285 22222 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:14.173409 31339 tablet.cc:1722] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:14.173949 31342 tablet.cc:1722] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:14.175997 31340 tablet.cc:1722] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:14.180187 22230 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=d00d64e385d249c9890fa7e6e3b83dbf] alter complete (version 3)
I20240502 07:59:14.247298 30954 raft_consensus.cc:482] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:14.247447 30954 raft_consensus.cc:504] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:14.247929 30954 leader_election.cc:288] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:14.248276 22499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89889b6805684edeb79afd3fa3a55dea" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:14.248282 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89889b6805684edeb79afd3fa3a55dea" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:14.248404 22499 raft_consensus.cc:2454] T 89889b6805684edeb79afd3fa3a55dea P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:14.248437 22764 raft_consensus.cc:2454] T 89889b6805684edeb79afd3fa3a55dea P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
W20240502 07:59:14.248703 22812 tablet.cc:2369] T 89889b6805684edeb79afd3fa3a55dea P f00f3a1876444ad2bf9c919068ab8014: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:59:14.248770 25148 leader_election.cc:302] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:14.248942 30954 raft_consensus.cc:2781] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:14.249006 30954 raft_consensus.cc:482] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:14.249047 30954 raft_consensus.cc:3037] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:14.250344 30954 raft_consensus.cc:504] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:14.250705 30954 leader_election.cc:288] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:14.251533 22499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89889b6805684edeb79afd3fa3a55dea" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:14.251559 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89889b6805684edeb79afd3fa3a55dea" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:14.251646 22499 raft_consensus.cc:3037] T 89889b6805684edeb79afd3fa3a55dea P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:14.251681 22764 raft_consensus.cc:3037] T 89889b6805684edeb79afd3fa3a55dea P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:14.252485 22764 raft_consensus.cc:2454] T 89889b6805684edeb79afd3fa3a55dea P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:14.252528 22499 raft_consensus.cc:2454] T 89889b6805684edeb79afd3fa3a55dea P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:14.252828 25148 leader_election.cc:302] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:14.252995 30954 raft_consensus.cc:2781] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:14.253088 30954 raft_consensus.cc:686] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:14.253245 30954 consensus_queue.cc:237] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:14.255589 22224 catalog_manager.cc:5484] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:14.281348 24800 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:14.283658 24932 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:14.283740 22224 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:14.283933 22224 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:14.291116 22482 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:14.291798 22614 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:14.292606 22745 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:14.292870 22352 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:14.296211 22224 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 9e053de314dd4a6bb109f77f7f3c2b5a
I20240502 07:59:14.296824 24932 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e053de314dd4a6bb109f77f7f3c2b5a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:14 UTC) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:14.296905 22614 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e053de314dd4a6bb109f77f7f3c2b5a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:14 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:14.297457 31462 tablet_replica.cc:330] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:14.297544 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e053de314dd4a6bb109f77f7f3c2b5a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:14 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:14.297657 31462 raft_consensus.cc:2229] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:14.297783 31462 raft_consensus.cc:2258] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:14.298875 31462 ts_tablet_manager.cc:1875] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:14.308573 31462 ts_tablet_manager.cc:1888] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:14.308779 31462 log.cc:1196] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/9e053de314dd4a6bb109f77f7f3c2b5a
I20240502 07:59:14.308993 25063 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:14.309370 31462 ts_tablet_manager.cc:1909] T 9e053de314dd4a6bb109f77f7f3c2b5a P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:59:14.309478 25193 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:14.310693 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 9e053de314dd4a6bb109f77f7f3c2b5a (table rebalancer_dynamic_table_new_name [id=d00d64e385d249c9890fa7e6e3b83dbf]) successfully deleted
I20240502 07:59:14.336258 31461 tablet_replica.cc:330] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:14.336448 31461 raft_consensus.cc:2229] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:14.337486 31461 raft_consensus.cc:2258] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:14.338420 31463 tablet_replica.cc:330] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:14.338678 31463 raft_consensus.cc:2229] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:14.338924 31463 raft_consensus.cc:2258] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:14.339813 31463 ts_tablet_manager.cc:1875] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:14.341440 31461 ts_tablet_manager.cc:1875] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:14.345142 31461 ts_tablet_manager.cc:1888] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:14.346942 31463 ts_tablet_manager.cc:1888] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:14.347028 31463 log.cc:1196] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/9e053de314dd4a6bb109f77f7f3c2b5a
I20240502 07:59:14.347399 31463 ts_tablet_manager.cc:1909] T 9e053de314dd4a6bb109f77f7f3c2b5a P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:59:14.348119 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 9e053de314dd4a6bb109f77f7f3c2b5a (table rebalancer_dynamic_table_new_name [id=d00d64e385d249c9890fa7e6e3b83dbf]) successfully deleted
I20240502 07:59:14.351780 31461 log.cc:1196] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/9e053de314dd4a6bb109f77f7f3c2b5a
I20240502 07:59:14.352154 31461 ts_tablet_manager.cc:1909] T 9e053de314dd4a6bb109f77f7f3c2b5a P 1aecef81733c47b3b468504934357231: Deleting consensus metadata
I20240502 07:59:14.352981 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet 9e053de314dd4a6bb109f77f7f3c2b5a (table rebalancer_dynamic_table_new_name [id=d00d64e385d249c9890fa7e6e3b83dbf]) successfully deleted
I20240502 07:59:14.396907 31485 ts_tablet_manager.cc:902] T f5f0690b92c140c49fe10ff46e0136df P 4b4e7eeb7f3f42119c0363a79febc54b: Initiating tablet copy from peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:14.397420 31485 tablet_copy_client.cc:308] T f5f0690b92c140c49fe10ff46e0136df P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.129:38535
I20240502 07:59:14.397959 22392 tablet_copy_service.cc:140] P b533a7d8d97943609c6e28e3e36b78a4: Received BeginTabletCopySession request for tablet f5f0690b92c140c49fe10ff46e0136df from peer 4b4e7eeb7f3f42119c0363a79febc54b ({username='slave'} at 127.15.129.133:34117)
I20240502 07:59:14.398059 22224 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:14.398065 22392 tablet_copy_service.cc:161] P b533a7d8d97943609c6e28e3e36b78a4: Beginning new tablet copy session on tablet f5f0690b92c140c49fe10ff46e0136df from peer 4b4e7eeb7f3f42119c0363a79febc54b at {username='slave'} at 127.15.129.133:34117: session id = 4b4e7eeb7f3f42119c0363a79febc54b-f5f0690b92c140c49fe10ff46e0136df
I20240502 07:59:14.399549 22392 tablet_copy_source_session.cc:205] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:14.400535 31485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5f0690b92c140c49fe10ff46e0136df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.403026 31485 tablet_copy_client.cc:783] T f5f0690b92c140c49fe10ff46e0136df P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:14.403177 31485 tablet_copy_client.cc:647] T f5f0690b92c140c49fe10ff46e0136df P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:14.405972 31485 tablet_copy_client.cc:523] T f5f0690b92c140c49fe10ff46e0136df P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:14.407364 31485 tablet_bootstrap.cc:492] T f5f0690b92c140c49fe10ff46e0136df P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:14.410506 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 04aed25a23904c3dbeeb004aaa667e9d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=416ffcc145f44766ad076aa156acff90]), partition=
I20240502 07:59:14.410530 22482 tablet_service.cc:1467] Processing CreateTablet for tablet 04aed25a23904c3dbeeb004aaa667e9d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=416ffcc145f44766ad076aa156acff90]), partition=
I20240502 07:59:14.410771 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04aed25a23904c3dbeeb004aaa667e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.410770 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04aed25a23904c3dbeeb004aaa667e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.412358 31400 tablet_bootstrap.cc:492] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:14.413043 31404 tablet_bootstrap.cc:492] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:14.413408 31400 tablet_bootstrap.cc:654] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.413867 31404 tablet_bootstrap.cc:654] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.414594 25193 tablet_service.cc:1467] Processing CreateTablet for tablet 04aed25a23904c3dbeeb004aaa667e9d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=416ffcc145f44766ad076aa156acff90]), partition=
I20240502 07:59:14.414841 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04aed25a23904c3dbeeb004aaa667e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.418193 31405 tablet_bootstrap.cc:492] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:14.419046 31405 tablet_bootstrap.cc:654] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.433804 31404 tablet_bootstrap.cc:492] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:14.433917 31404 ts_tablet_manager.cc:1367] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 07:59:14.434402 31404 raft_consensus.cc:348] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:14.434531 31404 raft_consensus.cc:374] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.434566 31404 raft_consensus.cc:729] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.434690 31404 consensus_queue.cc:260] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:14.434958 31404 ts_tablet_manager.cc:1398] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:14.436643 31400 tablet_bootstrap.cc:492] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:14.436748 31400 ts_tablet_manager.cc:1367] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.002s
I20240502 07:59:14.437239 31400 raft_consensus.cc:348] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:14.437383 31400 raft_consensus.cc:374] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.437417 31400 raft_consensus.cc:729] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.437522 31400 consensus_queue.cc:260] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:14.437772 31400 ts_tablet_manager.cc:1398] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:14.438285 31485 tablet_bootstrap.cc:492] T f5f0690b92c140c49fe10ff46e0136df P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:14.438589 31485 tablet_bootstrap.cc:492] T f5f0690b92c140c49fe10ff46e0136df P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap complete.
I20240502 07:59:14.438764 31485 ts_tablet_manager.cc:1367] T f5f0690b92c140c49fe10ff46e0136df P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.031s user 0.023s sys 0.004s
I20240502 07:59:14.439170 31485 raft_consensus.cc:348] T f5f0690b92c140c49fe10ff46e0136df P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:14.439280 31485 raft_consensus.cc:729] T f5f0690b92c140c49fe10ff46e0136df P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.439445 31485 consensus_queue.cc:260] T f5f0690b92c140c49fe10ff46e0136df P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:14.439812 31485 ts_tablet_manager.cc:1398] T f5f0690b92c140c49fe10ff46e0136df P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:14.440300 22392 tablet_copy_service.cc:342] P b533a7d8d97943609c6e28e3e36b78a4: Request end of tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-f5f0690b92c140c49fe10ff46e0136df received from {username='slave'} at 127.15.129.133:34117
I20240502 07:59:14.440375 22392 tablet_copy_service.cc:433] P b533a7d8d97943609c6e28e3e36b78a4: ending tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-f5f0690b92c140c49fe10ff46e0136df on tablet f5f0690b92c140c49fe10ff46e0136df with peer 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:14.445745 31405 tablet_bootstrap.cc:492] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:14.445858 31405 ts_tablet_manager.cc:1367] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20240502 07:59:14.446334 31405 raft_consensus.cc:348] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:14.446481 31405 raft_consensus.cc:374] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.446518 31405 raft_consensus.cc:729] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.446642 31405 consensus_queue.cc:260] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:14.446933 31405 ts_tablet_manager.cc:1398] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:14.543643 31325 raft_consensus.cc:482] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:14.543771 31325 raft_consensus.cc:504] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:14.544209 31325 leader_election.cc:288] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:14.544648 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04aed25a23904c3dbeeb004aaa667e9d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:14.544795 25083 raft_consensus.cc:2454] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:14.545085 22499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04aed25a23904c3dbeeb004aaa667e9d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:14.545239 22499 raft_consensus.cc:2454] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:14.545557 25148 leader_election.cc:302] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:14.546159 31325 raft_consensus.cc:2781] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:14.546222 31325 raft_consensus.cc:482] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:14.546265 31325 raft_consensus.cc:3037] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:14.547458 31325 raft_consensus.cc:504] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:14.547753 31325 leader_election.cc:288] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:14.548177 22499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04aed25a23904c3dbeeb004aaa667e9d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:14.548307 22499 raft_consensus.cc:3037] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:14.548404 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04aed25a23904c3dbeeb004aaa667e9d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:14.548579 25083 raft_consensus.cc:3037] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:14.549465 22499 raft_consensus.cc:2454] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:14.549645 25083 raft_consensus.cc:2454] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:14.549856 25148 leader_election.cc:302] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:14.550060 31325 raft_consensus.cc:2781] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:14.550151 31325 raft_consensus.cc:686] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:14.550329 31325 consensus_queue.cc:237] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:14.552651 22222 catalog_manager.cc:5484] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:14.557227 31421 consensus_queue.cc:1035] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:14.563472 30670 consensus_queue.cc:1035] T 5246ae54d56b419bbfef485e7bc18ab9 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:14.579910 25193 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:14.580770 24932 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:14.581983 24800 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:14.584051 25063 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:14.590932 22482 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:14.595767 22352 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:14.598091 31418 consensus_queue.cc:1035] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:14.606608 22614 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:14.611203 31203 consensus_queue.cc:1035] T 69f190895a344889b24868b3a2e4a41a P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:14.623896 22745 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:14.679925 22222 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:14.684923 25083 raft_consensus.cc:1261] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:14.685041 22499 raft_consensus.cc:1261] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:14.685390 31325 consensus_queue.cc:1035] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:14.685554 30954 consensus_queue.cc:1035] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:14.688650 22222 catalog_manager.cc:5576] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625) does not have the latest schema for tablet 04aed25a23904c3dbeeb004aaa667e9d (table rebalancer_dynamic_table [id=416ffcc145f44766ad076aa156acff90]). Expected version 1 got 0
I20240502 07:59:14.689509 22228 catalog_manager.cc:5576] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733) does not have the latest schema for tablet 04aed25a23904c3dbeeb004aaa667e9d (table rebalancer_dynamic_table [id=416ffcc145f44766ad076aa156acff90]). Expected version 1 got 0
I20240502 07:59:14.689940 31554 tablet.cc:1722] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:14.690274 31554 tablet.cc:1620] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:14.693033 22228 catalog_manager.cc:5576] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625) does not have the latest schema for tablet 04aed25a23904c3dbeeb004aaa667e9d (table rebalancer_dynamic_table [id=416ffcc145f44766ad076aa156acff90]). Expected version 1 got 0
I20240502 07:59:14.694375 31546 mvcc.cc:205] Tried to move back new op lower bound from 7023152147185606656 to 7023152146639597568. Current Snapshot: MvccSnapshot[applied={T|T < 7023152147185606656}]
I20240502 07:59:14.697072 31555 tablet.cc:1722] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:14.697499 31555 tablet.cc:1620] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c: MemRowSet was empty: no flush needed.
I20240502 07:59:14.698244 31530 mvcc.cc:205] Tried to move back new op lower bound from 7023152147185606656 to 7023152146639597568. Current Snapshot: MvccSnapshot[applied={T|T < 7023152147185606656}]
I20240502 07:59:14.697780 22499 consensus_queue.cc:237] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:14.699810 22222 catalog_manager.cc:5576] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733) does not have the latest schema for tablet 04aed25a23904c3dbeeb004aaa667e9d (table rebalancer_dynamic_table [id=416ffcc145f44766ad076aa156acff90]). Expected version 1 got 0
I20240502 07:59:14.700695 31556 tablet.cc:1722] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:14.700901 22763 raft_consensus.cc:1261] T a0430566f5ae4ffabcf15bb8b8af6bb5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:14.701022 31556 tablet.cc:1620] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:14.701277 31420 consensus_queue.cc:1035] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:14.701426 24820 raft_consensus.cc:1261] T a0430566f5ae4ffabcf15bb8b8af6bb5 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:14.701817 31420 consensus_queue.cc:1035] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:14.704070 22228 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=416ffcc145f44766ad076aa156acff90] alter complete (version 1)
I20240502 07:59:14.704924 30670 raft_consensus.cc:2932] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:14.705399 22763 raft_consensus.cc:2932] T a0430566f5ae4ffabcf15bb8b8af6bb5 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:14.707062 24820 raft_consensus.cc:2932] T a0430566f5ae4ffabcf15bb8b8af6bb5 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:14.708812 22222 catalog_manager.cc:5484] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c reported cstate change: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:14.712018 22614 tablet_service.cc:1514] Processing DeleteTablet for tablet a0430566f5ae4ffabcf15bb8b8af6bb5 with delete_type TABLET_DATA_TOMBSTONED (TS e51b10af49484aaa8f0267aebc1bf27f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:14.712275 31462 tablet_replica.cc:330] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:14.712442 31462 raft_consensus.cc:2229] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:14.712563 31462 raft_consensus.cc:2258] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:14.713081 31462 ts_tablet_manager.cc:1875] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:14.714879 31462 ts_tablet_manager.cc:1888] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:14.714939 31462 log.cc:1196] T a0430566f5ae4ffabcf15bb8b8af6bb5 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/a0430566f5ae4ffabcf15bb8b8af6bb5
I20240502 07:59:14.715466 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet a0430566f5ae4ffabcf15bb8b8af6bb5 (table rebalance_test_table_9 [id=9b69b771dcfb4a00a937ed9203245903]) successfully deleted
I20240502 07:59:14.754349 24951 consensus_queue.cc:237] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
W20240502 07:59:14.756364 24885 consensus_peers.cc:485] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 -> Peer fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): Couldn't send request to peer fabddb16c85944648a710ec52fc7244c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a3ffa0ad72d7493082f697a1215bdbf4. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:14.756384 24820 raft_consensus.cc:1261] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:14.756636 25213 raft_consensus.cc:1261] T a3ffa0ad72d7493082f697a1215bdbf4 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:14.756769 31130 consensus_queue.cc:1035] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:14.757088 31130 consensus_queue.cc:1035] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:14.759054 31130 raft_consensus.cc:2932] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:14.759529 25213 raft_consensus.cc:2932] T a3ffa0ad72d7493082f697a1215bdbf4 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:14.762106 24821 raft_consensus.cc:2932] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:14.762820 22228 catalog_manager.cc:5484] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 reported cstate change: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:14.767419 30954 consensus_queue.cc:1035] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:14.771090 30954 consensus_queue.cc:1035] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:14.773430 22222 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_46"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:14.781100 25193 tablet_service.cc:1467] Processing CreateTablet for tablet c6ef78ca89a34f7b94a58d8a6ea8b303 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=ae9e0c6dd5794a02bcfb8899f4d26273]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:14.781100 24932 tablet_service.cc:1467] Processing CreateTablet for tablet c6ef78ca89a34f7b94a58d8a6ea8b303 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=ae9e0c6dd5794a02bcfb8899f4d26273]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:14.781185 22745 tablet_service.cc:1467] Processing CreateTablet for tablet c6ef78ca89a34f7b94a58d8a6ea8b303 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=ae9e0c6dd5794a02bcfb8899f4d26273]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:14.781435 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6ef78ca89a34f7b94a58d8a6ea8b303. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.781435 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6ef78ca89a34f7b94a58d8a6ea8b303. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.781509 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6ef78ca89a34f7b94a58d8a6ea8b303. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.783699 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 02b326ae3ffa4afbb7a1dcfbad4c56ba (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=ae9e0c6dd5794a02bcfb8899f4d26273]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:14.783972 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02b326ae3ffa4afbb7a1dcfbad4c56ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.784221 24931 tablet_service.cc:1467] Processing CreateTablet for tablet 02b326ae3ffa4afbb7a1dcfbad4c56ba (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=ae9e0c6dd5794a02bcfb8899f4d26273]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:14.784350 31405 tablet_bootstrap.cc:492] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:14.784471 24931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02b326ae3ffa4afbb7a1dcfbad4c56ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.784971 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 02b326ae3ffa4afbb7a1dcfbad4c56ba (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=ae9e0c6dd5794a02bcfb8899f4d26273]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:14.785254 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02b326ae3ffa4afbb7a1dcfbad4c56ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.785315 31405 tablet_bootstrap.cc:654] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.786798 22614 tablet_service.cc:1467] Processing CreateTablet for tablet fe087f9b25c84749b7cfba34800da679 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=ae9e0c6dd5794a02bcfb8899f4d26273]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:14.786974 31405 tablet_bootstrap.cc:492] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:14.787071 31405 ts_tablet_manager.cc:1367] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:14.787096 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe087f9b25c84749b7cfba34800da679. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.787540 31607 tablet_bootstrap.cc:492] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:14.787542 31405 raft_consensus.cc:348] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:14.787663 31405 raft_consensus.cc:374] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.787694 31405 raft_consensus.cc:729] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.787807 31405 consensus_queue.cc:260] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:14.788069 31405 ts_tablet_manager.cc:1398] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:14.788161 31608 tablet_bootstrap.cc:492] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:14.788492 31607 tablet_bootstrap.cc:654] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.790127 31608 tablet_bootstrap.cc:654] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.790220 31604 tablet_bootstrap.cc:492] T c6ef78ca89a34f7b94a58d8a6ea8b303 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:14.784221 24930 tablet_service.cc:1467] Processing CreateTablet for tablet fe087f9b25c84749b7cfba34800da679 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=ae9e0c6dd5794a02bcfb8899f4d26273]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:14.791298 31604 tablet_bootstrap.cc:654] T c6ef78ca89a34f7b94a58d8a6ea8b303 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.791311 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe087f9b25c84749b7cfba34800da679. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.791685 31605 tablet_bootstrap.cc:492] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:14.792579 31605 tablet_bootstrap.cc:654] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.793761 31607 tablet_bootstrap.cc:492] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:14.793812 31605 tablet_bootstrap.cc:492] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:14.793841 31607 ts_tablet_manager.cc:1367] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:14.793895 31605 ts_tablet_manager.cc:1367] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:14.794252 31608 tablet_bootstrap.cc:492] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:14.794266 31607 raft_consensus.cc:348] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:14.794337 31608 ts_tablet_manager.cc:1367] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:14.794368 31607 raft_consensus.cc:374] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.794394 31607 raft_consensus.cc:729] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.794510 31607 consensus_queue.cc:260] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:14.794543 31605 raft_consensus.cc:348] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:14.794652 31605 raft_consensus.cc:374] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.794682 31605 raft_consensus.cc:729] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.794799 31605 consensus_queue.cc:260] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:14.795006 31608 raft_consensus.cc:348] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:14.795104 31608 raft_consensus.cc:374] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.795135 31608 raft_consensus.cc:729] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.795238 31608 consensus_queue.cc:260] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:14.795478 31607 ts_tablet_manager.cc:1398] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:14.795498 31608 ts_tablet_manager.cc:1398] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:14.799014 31605 ts_tablet_manager.cc:1398] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:59:14.800937 31605 tablet_bootstrap.cc:492] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:14.801970 31605 tablet_bootstrap.cc:654] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.803197 31605 tablet_bootstrap.cc:492] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:14.803279 31605 ts_tablet_manager.cc:1367] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:14.803489 22745 tablet_service.cc:1467] Processing CreateTablet for tablet fe087f9b25c84749b7cfba34800da679 (DEFAULT_TABLE table=rebalancer_extra_table_46 [id=ae9e0c6dd5794a02bcfb8899f4d26273]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:14.803731 31605 raft_consensus.cc:348] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:14.803784 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe087f9b25c84749b7cfba34800da679. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:14.803830 31605 raft_consensus.cc:374] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.803865 31605 raft_consensus.cc:729] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.803997 31605 consensus_queue.cc:260] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:14.804271 31605 ts_tablet_manager.cc:1398] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:14.804427 31605 tablet_bootstrap.cc:492] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:14.805410 31605 tablet_bootstrap.cc:654] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.806520 31605 tablet_bootstrap.cc:492] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:14.806649 31605 ts_tablet_manager.cc:1367] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:59:14.807092 31605 raft_consensus.cc:348] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:14.807193 31605 raft_consensus.cc:374] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.807245 31605 raft_consensus.cc:729] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.807227 22228 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:14.807988 31605 consensus_queue.cc:260] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:14.809092 31605 ts_tablet_manager.cc:1398] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:14.813807 31610 tablet_bootstrap.cc:492] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:14.814886 31610 tablet_bootstrap.cc:654] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.816428 31554 tablet.cc:1722] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:14.816730 31554 tablet.cc:1620] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:14.816867 31555 tablet.cc:1722] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:14.817138 31555 tablet.cc:1620] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c: MemRowSet was empty: no flush needed.
I20240502 07:59:14.820189 31604 tablet_bootstrap.cc:492] T c6ef78ca89a34f7b94a58d8a6ea8b303 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:14.820291 31604 ts_tablet_manager.cc:1367] T c6ef78ca89a34f7b94a58d8a6ea8b303 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20240502 07:59:14.821396 31604 raft_consensus.cc:348] T c6ef78ca89a34f7b94a58d8a6ea8b303 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:14.821507 31604 raft_consensus.cc:374] T c6ef78ca89a34f7b94a58d8a6ea8b303 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.821539 31604 raft_consensus.cc:729] T c6ef78ca89a34f7b94a58d8a6ea8b303 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.821657 31604 consensus_queue.cc:260] T c6ef78ca89a34f7b94a58d8a6ea8b303 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:14.821919 31604 ts_tablet_manager.cc:1398] T c6ef78ca89a34f7b94a58d8a6ea8b303 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:14.822057 31604 tablet_bootstrap.cc:492] T fe087f9b25c84749b7cfba34800da679 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:14.822775 31604 tablet_bootstrap.cc:654] T fe087f9b25c84749b7cfba34800da679 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:14.824335 31610 tablet_bootstrap.cc:492] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:14.826057 31604 tablet_bootstrap.cc:492] T fe087f9b25c84749b7cfba34800da679 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:14.826148 31604 ts_tablet_manager.cc:1367] T fe087f9b25c84749b7cfba34800da679 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:14.826550 31556 tablet.cc:1722] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:14.826663 31604 raft_consensus.cc:348] T fe087f9b25c84749b7cfba34800da679 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:14.826777 31604 raft_consensus.cc:374] T fe087f9b25c84749b7cfba34800da679 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.826820 31604 raft_consensus.cc:729] T fe087f9b25c84749b7cfba34800da679 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.826855 31556 tablet.cc:1620] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:14.826947 31604 consensus_queue.cc:260] T fe087f9b25c84749b7cfba34800da679 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:14.827212 31604 ts_tablet_manager.cc:1398] T fe087f9b25c84749b7cfba34800da679 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:14.824433 31610 ts_tablet_manager.cc:1367] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 07:59:14.829198 31610 raft_consensus.cc:348] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:14.829381 31610 raft_consensus.cc:374] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:14.829478 31610 raft_consensus.cc:729] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:14.830173 31610 consensus_queue.cc:260] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:14.831923 22222 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=416ffcc145f44766ad076aa156acff90] alter complete (version 2)
I20240502 07:59:14.835877 31610 ts_tablet_manager.cc:1398] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20240502 07:59:14.914544 31586 raft_consensus.cc:482] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:14.914669 31586 raft_consensus.cc:504] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:14.915135 31586 leader_election.cc:288] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:14.915671 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ef78ca89a34f7b94a58d8a6ea8b303" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:14.915688 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ef78ca89a34f7b94a58d8a6ea8b303" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:14.915798 22763 raft_consensus.cc:2454] T c6ef78ca89a34f7b94a58d8a6ea8b303 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:14.915822 25213 raft_consensus.cc:2454] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:14.916054 24888 leader_election.cc:302] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:14.916225 31586 raft_consensus.cc:2781] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:14.916276 31586 raft_consensus.cc:482] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:14.916309 31586 raft_consensus.cc:3037] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:14.917376 31586 raft_consensus.cc:504] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:14.917690 31586 leader_election.cc:288] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:14.918009 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ef78ca89a34f7b94a58d8a6ea8b303" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:14.918112 25213 raft_consensus.cc:3037] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:14.918272 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6ef78ca89a34f7b94a58d8a6ea8b303" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:14.918404 22763 raft_consensus.cc:3037] T c6ef78ca89a34f7b94a58d8a6ea8b303 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:14.918908 25213 raft_consensus.cc:2454] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:14.919184 24888 leader_election.cc:302] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:14.919281 22763 raft_consensus.cc:2454] T c6ef78ca89a34f7b94a58d8a6ea8b303 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:14.919564 31586 raft_consensus.cc:2781] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:14.919718 31586 raft_consensus.cc:686] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:14.919852 31586 consensus_queue.cc:237] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:14.922065 22230 catalog_manager.cc:5484] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:14.934605 24821 raft_consensus.cc:1203] T f5f0690b92c140c49fe10ff46e0136df P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:14.935582 22230 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:14.941200 31554 tablet.cc:1722] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:14.943228 31556 tablet.cc:1722] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:14.945075 31555 tablet.cc:1722] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:14.946938 22230 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=416ffcc145f44766ad076aa156acff90] alter complete (version 3)
I20240502 07:59:14.999845 29808 raft_consensus.cc:482] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:14.999961 29808 raft_consensus.cc:504] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:15.000388 29808 leader_election.cc:288] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:15.000722 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02b326ae3ffa4afbb7a1dcfbad4c56ba" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:15.000737 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02b326ae3ffa4afbb7a1dcfbad4c56ba" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:15.000861 24951 raft_consensus.cc:2454] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:15.000861 24821 raft_consensus.cc:2454] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:15.001142 22307 leader_election.cc:302] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:15.001287 29808 raft_consensus.cc:2781] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:15.001360 29808 raft_consensus.cc:482] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:15.001413 29808 raft_consensus.cc:3037] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:15.002465 29808 raft_consensus.cc:504] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:15.002839 29808 leader_election.cc:288] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:15.003255 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02b326ae3ffa4afbb7a1dcfbad4c56ba" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:15.003263 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02b326ae3ffa4afbb7a1dcfbad4c56ba" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:15.003358 24951 raft_consensus.cc:3037] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:15.003388 24820 raft_consensus.cc:3037] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:15.004160 24820 raft_consensus.cc:2454] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:15.004158 24951 raft_consensus.cc:2454] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:15.004406 22308 leader_election.cc:302] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:15.004638 29808 raft_consensus.cc:2781] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:15.004731 29808 raft_consensus.cc:686] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:59:15.005057 29808 consensus_queue.cc:237] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:15.007213 22228 catalog_manager.cc:5484] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:15.012960 31410 raft_consensus.cc:482] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:15.013084 31410 raft_consensus.cc:504] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:15.013615 31410 leader_election.cc:288] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:15.013957 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe087f9b25c84749b7cfba34800da679" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:15.013968 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe087f9b25c84749b7cfba34800da679" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:15.014087 22763 raft_consensus.cc:2454] T fe087f9b25c84749b7cfba34800da679 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:15.014091 24951 raft_consensus.cc:2454] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:15.014334 22570 leader_election.cc:302] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:15.014479 31410 raft_consensus.cc:2781] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:15.014533 31410 raft_consensus.cc:482] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:15.014575 31410 raft_consensus.cc:3037] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:15.015537 31410 raft_consensus.cc:504] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:15.015863 31410 leader_election.cc:288] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:15.016124 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe087f9b25c84749b7cfba34800da679" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:15.016237 24951 raft_consensus.cc:3037] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:15.016220 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe087f9b25c84749b7cfba34800da679" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:15.016322 22763 raft_consensus.cc:3037] T fe087f9b25c84749b7cfba34800da679 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:15.016990 24951 raft_consensus.cc:2454] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:15.017151 22763 raft_consensus.cc:2454] T fe087f9b25c84749b7cfba34800da679 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:15.017457 22570 leader_election.cc:302] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:15.017663 31410 raft_consensus.cc:2781] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:15.017782 31410 raft_consensus.cc:686] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:59:15.017951 31410 consensus_queue.cc:237] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:15.020553 22228 catalog_manager.cc:5484] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:15.044168 24800 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:15.046640 24930 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:15.050503 22228 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:15.050654 22228 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:15.052899 22228 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 04aed25a23904c3dbeeb004aaa667e9d
I20240502 07:59:15.053299 25193 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:15.053597 25194 tablet_service.cc:1514] Processing DeleteTablet for tablet 04aed25a23904c3dbeeb004aaa667e9d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:15 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:15.053968 22482 tablet_service.cc:1514] Processing DeleteTablet for tablet 04aed25a23904c3dbeeb004aaa667e9d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:15 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:15.054289 31664 tablet_replica.cc:330] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:15.054500 31664 raft_consensus.cc:2229] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:15.055652 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet 04aed25a23904c3dbeeb004aaa667e9d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:15 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:15.055783 31664 raft_consensus.cc:2258] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:15.056005 25062 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:15.057245 31664 ts_tablet_manager.cc:1875] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:15.059653 31664 ts_tablet_manager.cc:1888] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:15.059760 31664 log.cc:1196] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/04aed25a23904c3dbeeb004aaa667e9d
I20240502 07:59:15.060096 31664 ts_tablet_manager.cc:1909] T 04aed25a23904c3dbeeb004aaa667e9d P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:59:15.060789 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 04aed25a23904c3dbeeb004aaa667e9d (table rebalancer_dynamic_table_new_name [id=416ffcc145f44766ad076aa156acff90]) successfully deleted
I20240502 07:59:15.061477 31663 tablet_replica.cc:330] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:15.061504 31665 tablet_replica.cc:330] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:15.061637 31665 raft_consensus.cc:2229] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:15.061669 31663 raft_consensus.cc:2229] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:15.061755 31665 raft_consensus.cc:2258] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:15.061949 31663 raft_consensus.cc:2258] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:15.062546 31663 ts_tablet_manager.cc:1875] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:15.062579 31665 ts_tablet_manager.cc:1875] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:15.064675 31665 ts_tablet_manager.cc:1888] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:15.064747 31665 log.cc:1196] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/04aed25a23904c3dbeeb004aaa667e9d
I20240502 07:59:15.064994 31665 ts_tablet_manager.cc:1909] T 04aed25a23904c3dbeeb004aaa667e9d P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:59:15.069679 31663 ts_tablet_manager.cc:1888] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:15.070472 31663 log.cc:1196] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/04aed25a23904c3dbeeb004aaa667e9d
I20240502 07:59:15.070822 31663 ts_tablet_manager.cc:1909] T 04aed25a23904c3dbeeb004aaa667e9d P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:59:15.073472 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet 04aed25a23904c3dbeeb004aaa667e9d (table rebalancer_dynamic_table_new_name [id=416ffcc145f44766ad076aa156acff90]) successfully deleted
I20240502 07:59:15.073701 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 04aed25a23904c3dbeeb004aaa667e9d (table rebalancer_dynamic_table_new_name [id=416ffcc145f44766ad076aa156acff90]) successfully deleted
I20240502 07:59:15.076545 22614 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 07:59:15.078961 22352 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:15.081262 22745 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:15.118506 22482 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:15.155014 22228 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:15.161716 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 3d5511d63ba84b6d948d303701f23f7f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d3d4077a11f2475ea6143fb721adca37]), partition=
I20240502 07:59:15.161835 22614 tablet_service.cc:1467] Processing CreateTablet for tablet 3d5511d63ba84b6d948d303701f23f7f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d3d4077a11f2475ea6143fb721adca37]), partition=
I20240502 07:59:15.161844 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 3d5511d63ba84b6d948d303701f23f7f (DEFAULT_TABLE table=rebalancer_dynamic_table [id=d3d4077a11f2475ea6143fb721adca37]), partition=
I20240502 07:59:15.162014 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d5511d63ba84b6d948d303701f23f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:15.162056 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d5511d63ba84b6d948d303701f23f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:15.162079 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d5511d63ba84b6d948d303701f23f7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:15.164222 31607 tablet_bootstrap.cc:492] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:15.164227 31604 tablet_bootstrap.cc:492] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:15.165016 31604 tablet_bootstrap.cc:654] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:15.165128 31607 tablet_bootstrap.cc:654] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:15.165962 31610 tablet_bootstrap.cc:492] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:15.166922 31610 tablet_bootstrap.cc:654] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:15.167292 31607 tablet_bootstrap.cc:492] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:15.167366 31607 ts_tablet_manager.cc:1367] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:15.167932 31607 raft_consensus.cc:348] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:15.168043 31607 raft_consensus.cc:374] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:15.168071 31607 raft_consensus.cc:729] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:15.168187 31607 consensus_queue.cc:260] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:15.168488 31607 ts_tablet_manager.cc:1398] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:15.168725 31604 tablet_bootstrap.cc:492] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:15.168800 31604 ts_tablet_manager.cc:1367] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:15.169229 31604 raft_consensus.cc:348] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:15.169353 31604 raft_consensus.cc:374] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:15.169389 31604 raft_consensus.cc:729] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:15.169507 31604 consensus_queue.cc:260] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:15.169791 31604 ts_tablet_manager.cc:1398] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:15.170421 31610 tablet_bootstrap.cc:492] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:15.170508 31610 ts_tablet_manager.cc:1367] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:15.170981 31610 raft_consensus.cc:348] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:15.171084 31610 raft_consensus.cc:374] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:15.171123 31610 raft_consensus.cc:729] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:15.171249 31610 consensus_queue.cc:260] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:15.171521 31610 ts_tablet_manager.cc:1398] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:15.238188 31711 ts_tablet_manager.cc:902] T a3ffa0ad72d7493082f697a1215bdbf4 P fabddb16c85944648a710ec52fc7244c: Initiating tablet copy from peer 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:15.238781 31711 tablet_copy_client.cc:308] T a3ffa0ad72d7493082f697a1215bdbf4 P fabddb16c85944648a710ec52fc7244c: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.134:35995
I20240502 07:59:15.241533 24972 tablet_copy_service.cc:140] P 1aecef81733c47b3b468504934357231: Received BeginTabletCopySession request for tablet a3ffa0ad72d7493082f697a1215bdbf4 from peer fabddb16c85944648a710ec52fc7244c ({username='slave'} at 127.15.129.130:52777)
I20240502 07:59:15.241645 24972 tablet_copy_service.cc:161] P 1aecef81733c47b3b468504934357231: Beginning new tablet copy session on tablet a3ffa0ad72d7493082f697a1215bdbf4 from peer fabddb16c85944648a710ec52fc7244c at {username='slave'} at 127.15.129.130:52777: session id = fabddb16c85944648a710ec52fc7244c-a3ffa0ad72d7493082f697a1215bdbf4
I20240502 07:59:15.242637 24972 tablet_copy_source_session.cc:205] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:15.246125 31711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3ffa0ad72d7493082f697a1215bdbf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:15.251693 31711 tablet_copy_client.cc:783] T a3ffa0ad72d7493082f697a1215bdbf4 P fabddb16c85944648a710ec52fc7244c: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:15.253180 31711 tablet_copy_client.cc:647] T a3ffa0ad72d7493082f697a1215bdbf4 P fabddb16c85944648a710ec52fc7244c: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:15.254452 31711 tablet_copy_client.cc:523] T a3ffa0ad72d7493082f697a1215bdbf4 P fabddb16c85944648a710ec52fc7244c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:15.256583 31711 tablet_bootstrap.cc:492] T a3ffa0ad72d7493082f697a1215bdbf4 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:15.260272 31711 tablet_bootstrap.cc:492] T a3ffa0ad72d7493082f697a1215bdbf4 P fabddb16c85944648a710ec52fc7244c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:15.260499 31711 tablet_bootstrap.cc:492] T a3ffa0ad72d7493082f697a1215bdbf4 P fabddb16c85944648a710ec52fc7244c: Bootstrap complete.
I20240502 07:59:15.260656 31711 ts_tablet_manager.cc:1367] T a3ffa0ad72d7493082f697a1215bdbf4 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20240502 07:59:15.261039 31711 raft_consensus.cc:348] T a3ffa0ad72d7493082f697a1215bdbf4 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:15.261135 31711 raft_consensus.cc:729] T a3ffa0ad72d7493082f697a1215bdbf4 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:15.261292 31711 consensus_queue.cc:260] T a3ffa0ad72d7493082f697a1215bdbf4 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:15.261624 31711 ts_tablet_manager.cc:1398] T a3ffa0ad72d7493082f697a1215bdbf4 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240502 07:59:15.262830 24972 tablet_copy_service.cc:342] P 1aecef81733c47b3b468504934357231: Request end of tablet copy session fabddb16c85944648a710ec52fc7244c-a3ffa0ad72d7493082f697a1215bdbf4 received from {username='slave'} at 127.15.129.130:52777
I20240502 07:59:15.262919 24972 tablet_copy_service.cc:433] P 1aecef81733c47b3b468504934357231: ending tablet copy session fabddb16c85944648a710ec52fc7244c-a3ffa0ad72d7493082f697a1215bdbf4 on tablet a3ffa0ad72d7493082f697a1215bdbf4 with peer fabddb16c85944648a710ec52fc7244c
I20240502 07:59:15.410651 30776 raft_consensus.cc:482] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:15.410771 30776 raft_consensus.cc:504] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:15.411223 30776 leader_election.cc:288] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:15.411573 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d5511d63ba84b6d948d303701f23f7f" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:15.411633 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d5511d63ba84b6d948d303701f23f7f" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:15.411703 22372 raft_consensus.cc:2454] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:15.411759 22764 raft_consensus.cc:2454] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:15.412057 22570 leader_election.cc:302] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:15.412248 30776 raft_consensus.cc:2781] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:15.412304 30776 raft_consensus.cc:482] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:15.412355 30776 raft_consensus.cc:3037] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:15.413349 30776 raft_consensus.cc:504] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:15.413713 30776 leader_election.cc:288] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:15.414265 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d5511d63ba84b6d948d303701f23f7f" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:15.414393 22372 raft_consensus.cc:3037] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:15.415323 22372 raft_consensus.cc:2454] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:15.416026 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d5511d63ba84b6d948d303701f23f7f" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:15.416105 22570 leader_election.cc:302] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:15.416180 22764 raft_consensus.cc:3037] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:15.417184 22764 raft_consensus.cc:2454] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:15.418540 30776 raft_consensus.cc:2781] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:15.418639 30776 raft_consensus.cc:686] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:59:15.418787 30776 consensus_queue.cc:237] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:15.420814 22227 catalog_manager.cc:5484] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:15.421419 31589 consensus_queue.cc:1035] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:15.427645 31586 consensus_queue.cc:1035] T c6ef78ca89a34f7b94a58d8a6ea8b303 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:15.447311 24930 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:15.448611 25062 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:15.449275 24800 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:15.450451 25194 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:15.471983 22614 tablet_service.cc:1429] Tablet server has 22 leaders and 0 scanners
I20240502 07:59:15.472879 22352 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:15.475409 22482 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:15.493906 22745 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:15.508433 30776 consensus_queue.cc:1035] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:15.511379 30776 consensus_queue.cc:1035] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:15.535600 22226 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:15.546373 24930 tablet_service.cc:1467] Processing CreateTablet for tablet c951fb6e89f24b24967ce84b32e7330a (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=98a593719d064c9388b2d018ca117e71]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:15.546576 22352 tablet_service.cc:1467] Processing CreateTablet for tablet b422c91ffe87402c91a0bc58a0715940 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=98a593719d064c9388b2d018ca117e71]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:15.546716 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c951fb6e89f24b24967ce84b32e7330a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:15.546852 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b422c91ffe87402c91a0bc58a0715940. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:15.547019 22614 tablet_service.cc:1467] Processing CreateTablet for tablet b422c91ffe87402c91a0bc58a0715940 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=98a593719d064c9388b2d018ca117e71]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:15.547330 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b422c91ffe87402c91a0bc58a0715940. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:15.548727 31742 tablet_bootstrap.cc:492] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:15.549090 31607 tablet_bootstrap.cc:492] T b422c91ffe87402c91a0bc58a0715940 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:15.549572 31610 tablet_bootstrap.cc:492] T b422c91ffe87402c91a0bc58a0715940 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:15.550124 31607 tablet_bootstrap.cc:654] T b422c91ffe87402c91a0bc58a0715940 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:15.550544 31610 tablet_bootstrap.cc:654] T b422c91ffe87402c91a0bc58a0715940 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:15.550995 31742 tablet_bootstrap.cc:654] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:15.551832 24800 tablet_service.cc:1467] Processing CreateTablet for tablet b27345a5a0874524855f57f8903afe6c (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=98a593719d064c9388b2d018ca117e71]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:15.552150 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b27345a5a0874524855f57f8903afe6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:15.552361 31607 tablet_bootstrap.cc:492] T b422c91ffe87402c91a0bc58a0715940 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:15.552457 31607 ts_tablet_manager.cc:1367] T b422c91ffe87402c91a0bc58a0715940 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:15.552886 31607 raft_consensus.cc:348] T b422c91ffe87402c91a0bc58a0715940 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:15.552987 31607 raft_consensus.cc:374] T b422c91ffe87402c91a0bc58a0715940 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:15.553017 31607 raft_consensus.cc:729] T b422c91ffe87402c91a0bc58a0715940 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:15.553144 31607 consensus_queue.cc:260] T b422c91ffe87402c91a0bc58a0715940 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:15.553481 31607 ts_tablet_manager.cc:1398] T b422c91ffe87402c91a0bc58a0715940 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:15.554497 31747 tablet_bootstrap.cc:492] T b27345a5a0874524855f57f8903afe6c P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:15.554673 25194 tablet_service.cc:1467] Processing CreateTablet for tablet b27345a5a0874524855f57f8903afe6c (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=98a593719d064c9388b2d018ca117e71]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:15.554960 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b27345a5a0874524855f57f8903afe6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:15.555333 31747 tablet_bootstrap.cc:654] T b27345a5a0874524855f57f8903afe6c P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:15.556463 31747 tablet_bootstrap.cc:492] T b27345a5a0874524855f57f8903afe6c P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:15.556522 31747 ts_tablet_manager.cc:1367] T b27345a5a0874524855f57f8903afe6c P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:15.556856 31747 raft_consensus.cc:348] T b27345a5a0874524855f57f8903afe6c P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:15.556957 31747 raft_consensus.cc:374] T b27345a5a0874524855f57f8903afe6c P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:15.556989 31747 raft_consensus.cc:729] T b27345a5a0874524855f57f8903afe6c P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:15.557103 31747 consensus_queue.cc:260] T b27345a5a0874524855f57f8903afe6c P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:15.557449 31747 ts_tablet_manager.cc:1398] T b27345a5a0874524855f57f8903afe6c P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:15.560249 25062 tablet_service.cc:1467] Processing CreateTablet for tablet c951fb6e89f24b24967ce84b32e7330a (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=98a593719d064c9388b2d018ca117e71]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:15.560523 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c951fb6e89f24b24967ce84b32e7330a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:15.560474 22483 tablet_service.cc:1467] Processing CreateTablet for tablet b422c91ffe87402c91a0bc58a0715940 (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=98a593719d064c9388b2d018ca117e71]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:15.560474 22481 tablet_service.cc:1467] Processing CreateTablet for tablet b27345a5a0874524855f57f8903afe6c (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=98a593719d064c9388b2d018ca117e71]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:15.560755 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b422c91ffe87402c91a0bc58a0715940. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:15.560894 29808 consensus_queue.cc:1035] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:15.561128 22482 tablet_service.cc:1467] Processing CreateTablet for tablet c951fb6e89f24b24967ce84b32e7330a (DEFAULT_TABLE table=rebalancer_extra_table_48 [id=98a593719d064c9388b2d018ca117e71]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:15.561336 22481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b27345a5a0874524855f57f8903afe6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:15.561486 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c951fb6e89f24b24967ce84b32e7330a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:15.564571 31750 tablet_bootstrap.cc:492] T c951fb6e89f24b24967ce84b32e7330a P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:15.565593 31749 tablet_bootstrap.cc:492] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:15.565644 22227 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:15.565857 29808 consensus_queue.cc:1035] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:15.567426 31610 tablet_bootstrap.cc:492] T b422c91ffe87402c91a0bc58a0715940 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:15.567524 31610 ts_tablet_manager.cc:1367] T b422c91ffe87402c91a0bc58a0715940 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 07:59:15.567997 31610 raft_consensus.cc:348] T b422c91ffe87402c91a0bc58a0715940 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:15.568113 31610 raft_consensus.cc:374] T b422c91ffe87402c91a0bc58a0715940 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:15.568151 31610 raft_consensus.cc:729] T b422c91ffe87402c91a0bc58a0715940 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:15.568259 31610 consensus_queue.cc:260] T b422c91ffe87402c91a0bc58a0715940 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:15.568539 31610 ts_tablet_manager.cc:1398] T b422c91ffe87402c91a0bc58a0715940 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:15.570430 31752 tablet_bootstrap.cc:492] T c951fb6e89f24b24967ce84b32e7330a P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:15.570708 31749 tablet_bootstrap.cc:654] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:15.572878 22764 raft_consensus.cc:1261] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:15.573216 22372 raft_consensus.cc:1261] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:15.573717 31750 tablet_bootstrap.cc:654] T c951fb6e89f24b24967ce84b32e7330a P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:15.574249 31749 tablet_bootstrap.cc:492] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:15.574395 31749 ts_tablet_manager.cc:1367] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 07:59:15.575068 31749 raft_consensus.cc:348] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:15.575361 31749 raft_consensus.cc:374] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:15.575443 31749 raft_consensus.cc:729] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:15.575613 31750 tablet_bootstrap.cc:492] T c951fb6e89f24b24967ce84b32e7330a P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:15.575640 31749 consensus_queue.cc:260] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:15.575752 31750 ts_tablet_manager.cc:1367] T c951fb6e89f24b24967ce84b32e7330a P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 07:59:15.576218 31410 consensus_queue.cc:1035] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:15.576350 31717 consensus_queue.cc:1035] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:15.576841 31749 ts_tablet_manager.cc:1398] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:15.578687 31750 raft_consensus.cc:348] T c951fb6e89f24b24967ce84b32e7330a P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:15.578859 31750 raft_consensus.cc:374] T c951fb6e89f24b24967ce84b32e7330a P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:15.578933 31750 raft_consensus.cc:729] T c951fb6e89f24b24967ce84b32e7330a P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:15.579228 31750 consensus_queue.cc:260] T c951fb6e89f24b24967ce84b32e7330a P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:15.579593 31750 ts_tablet_manager.cc:1398] T c951fb6e89f24b24967ce84b32e7330a P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:15.580924 31750 tablet_bootstrap.cc:492] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:15.582113 31750 tablet_bootstrap.cc:654] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:15.584671 31742 tablet_bootstrap.cc:492] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:15.584775 31742 ts_tablet_manager.cc:1367] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.000s
I20240502 07:59:15.585304 31742 raft_consensus.cc:348] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:15.585465 31742 raft_consensus.cc:374] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:15.585500 31742 raft_consensus.cc:729] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:15.585628 31742 consensus_queue.cc:260] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:15.585952 31742 ts_tablet_manager.cc:1398] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:15.588398 22222 catalog_manager.cc:5576] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673) does not have the latest schema for tablet 3d5511d63ba84b6d948d303701f23f7f (table rebalancer_dynamic_table [id=d3d4077a11f2475ea6143fb721adca37]). Expected version 1 got 0
I20240502 07:59:15.589422 22226 catalog_manager.cc:5576] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535) does not have the latest schema for tablet 3d5511d63ba84b6d948d303701f23f7f (table rebalancer_dynamic_table [id=d3d4077a11f2475ea6143fb721adca37]). Expected version 1 got 0
I20240502 07:59:15.591821 31752 tablet_bootstrap.cc:654] T c951fb6e89f24b24967ce84b32e7330a P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:15.592907 31752 tablet_bootstrap.cc:492] T c951fb6e89f24b24967ce84b32e7330a P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:15.593048 31752 ts_tablet_manager.cc:1367] T c951fb6e89f24b24967ce84b32e7330a P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.000s
I20240502 07:59:15.594619 31750 tablet_bootstrap.cc:492] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:15.594719 31750 ts_tablet_manager.cc:1367] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 07:59:15.595441 31750 raft_consensus.cc:348] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:15.595548 31752 raft_consensus.cc:348] T c951fb6e89f24b24967ce84b32e7330a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:15.595651 31750 raft_consensus.cc:374] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:15.595706 31752 raft_consensus.cc:374] T c951fb6e89f24b24967ce84b32e7330a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:15.595743 31752 raft_consensus.cc:729] T c951fb6e89f24b24967ce84b32e7330a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:15.595777 31750 raft_consensus.cc:729] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:15.595890 31754 mvcc.cc:205] Tried to move back new op lower bound from 7023152150820573184 to 7023152150196699136. Current Snapshot: MvccSnapshot[applied={T|T < 7023152150820573184}]
I20240502 07:59:15.596056 31750 consensus_queue.cc:260] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:15.596132 31752 consensus_queue.cc:260] T c951fb6e89f24b24967ce84b32e7330a P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:15.596155 31769 tablet.cc:1722] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:15.596467 31769 tablet.cc:1620] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:15.596604 31750 ts_tablet_manager.cc:1398] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:15.596638 31752 ts_tablet_manager.cc:1398] T c951fb6e89f24b24967ce84b32e7330a P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:15.596922 31750 tablet_bootstrap.cc:492] T b27345a5a0874524855f57f8903afe6c P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:15.597839 22226 catalog_manager.cc:5576] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535) does not have the latest schema for tablet 3d5511d63ba84b6d948d303701f23f7f (table rebalancer_dynamic_table [id=d3d4077a11f2475ea6143fb721adca37]). Expected version 1 got 0
I20240502 07:59:15.597963 31750 tablet_bootstrap.cc:654] T b27345a5a0874524855f57f8903afe6c P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:15.598852 31768 tablet.cc:1722] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:15.599222 31768 tablet.cc:1620] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
I20240502 07:59:15.601240 22222 catalog_manager.cc:5576] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673) does not have the latest schema for tablet 3d5511d63ba84b6d948d303701f23f7f (table rebalancer_dynamic_table [id=d3d4077a11f2475ea6143fb721adca37]). Expected version 1 got 0
I20240502 07:59:15.606052 31771 tablet.cc:1722] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:15.606365 31771 tablet.cc:1620] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:59:15.606812 22229 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=d3d4077a11f2475ea6143fb721adca37] alter complete (version 1)
I20240502 07:59:15.613834 31750 tablet_bootstrap.cc:492] T b27345a5a0874524855f57f8903afe6c P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:15.613946 31750 ts_tablet_manager.cc:1367] T b27345a5a0874524855f57f8903afe6c P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.002s
I20240502 07:59:15.614377 31750 raft_consensus.cc:348] T b27345a5a0874524855f57f8903afe6c P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:15.614511 31750 raft_consensus.cc:374] T b27345a5a0874524855f57f8903afe6c P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:15.614540 31750 raft_consensus.cc:729] T b27345a5a0874524855f57f8903afe6c P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:15.614650 31750 consensus_queue.cc:260] T b27345a5a0874524855f57f8903afe6c P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:15.615763 31750 ts_tablet_manager.cc:1398] T b27345a5a0874524855f57f8903afe6c P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:59:15.627427 22370 consensus_queue.cc:237] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:15.628801 24821 raft_consensus.cc:1261] T f5f0690b92c140c49fe10ff46e0136df P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:15.629238 31617 consensus_queue.cc:1035] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:15.629500 22500 raft_consensus.cc:1261] T f5f0690b92c140c49fe10ff46e0136df P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:15.630681 31617 consensus_queue.cc:1035] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:15.631460 31617 raft_consensus.cc:2932] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:15.631789 24821 raft_consensus.cc:2932] T f5f0690b92c140c49fe10ff46e0136df P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:15.632740 22500 raft_consensus.cc:2932] T f5f0690b92c140c49fe10ff46e0136df P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:15.635694 22226 catalog_manager.cc:5484] T f5f0690b92c140c49fe10ff46e0136df P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:15.640213 22614 tablet_service.cc:1514] Processing DeleteTablet for tablet f5f0690b92c140c49fe10ff46e0136df with delete_type TABLET_DATA_TOMBSTONED (TS e51b10af49484aaa8f0267aebc1bf27f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:15.640586 31786 tablet_replica.cc:330] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:15.640738 31786 raft_consensus.cc:2229] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:15.640885 31786 raft_consensus.cc:2258] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:15.641420 31786 ts_tablet_manager.cc:1875] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:15.643942 31786 ts_tablet_manager.cc:1888] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:15.644044 31786 log.cc:1196] T f5f0690b92c140c49fe10ff46e0136df P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/f5f0690b92c140c49fe10ff46e0136df
I20240502 07:59:15.644639 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet f5f0690b92c140c49fe10ff46e0136df (table rebalance_test_table_3 [id=2c887f37ddf349f3bbec2b1330f05fc7]) successfully deleted
I20240502 07:59:15.664371 22372 consensus_queue.cc:237] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
W20240502 07:59:15.665999 22307 consensus_peers.cc:485] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 423d6be23b044130b0c916f995dffced. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:15.665999 22632 raft_consensus.cc:1261] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:15.666460 31755 consensus_queue.cc:1035] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:15.666612 22764 raft_consensus.cc:1261] T 423d6be23b044130b0c916f995dffced P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:15.667053 31755 consensus_queue.cc:1035] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:15.668979 31617 raft_consensus.cc:2932] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:15.669509 22764 raft_consensus.cc:2932] T 423d6be23b044130b0c916f995dffced P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:15.673036 22229 catalog_manager.cc:5484] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:15.673277 22632 raft_consensus.cc:2932] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:15.679293 31586 raft_consensus.cc:482] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:15.679391 31586 raft_consensus.cc:504] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:15.679814 31586 leader_election.cc:288] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:15.680222 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c951fb6e89f24b24967ce84b32e7330a" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:15.680374 25084 raft_consensus.cc:2454] T c951fb6e89f24b24967ce84b32e7330a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:15.680349 22500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c951fb6e89f24b24967ce84b32e7330a" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:15.680490 22500 raft_consensus.cc:2454] T c951fb6e89f24b24967ce84b32e7330a P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:15.681005 24885 leader_election.cc:302] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
I20240502 07:59:15.681185 31589 raft_consensus.cc:2781] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:15.681252 31589 raft_consensus.cc:482] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:15.681289 31589 raft_consensus.cc:3037] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:15.682363 31589 raft_consensus.cc:504] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:15.682724 31589 leader_election.cc:288] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:15.683020 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c951fb6e89f24b24967ce84b32e7330a" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:15.683137 25084 raft_consensus.cc:3037] T c951fb6e89f24b24967ce84b32e7330a P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:15.683972 25084 raft_consensus.cc:2454] T c951fb6e89f24b24967ce84b32e7330a P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:15.684214 22500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c951fb6e89f24b24967ce84b32e7330a" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:15.684330 22500 raft_consensus.cc:3037] T c951fb6e89f24b24967ce84b32e7330a P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:15.684307 24885 leader_election.cc:302] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
I20240502 07:59:15.684478 31589 raft_consensus.cc:2781] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:15.684576 31589 raft_consensus.cc:686] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:15.684697 31589 consensus_queue.cc:237] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:15.685418 22500 raft_consensus.cc:2454] T c951fb6e89f24b24967ce84b32e7330a P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:15.687222 22229 catalog_manager.cc:5484] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:15.709470 22229 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:15.714672 31768 tablet.cc:1722] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:15.715019 31768 tablet.cc:1620] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
I20240502 07:59:15.715005 31769 tablet.cc:1722] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:15.715337 31769 tablet.cc:1620] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:15.715606 31771 tablet.cc:1722] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:15.715857 31771 tablet.cc:1620] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:59:15.723058 22229 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=d3d4077a11f2475ea6143fb721adca37] alter complete (version 2)
I20240502 07:59:15.735862 31420 raft_consensus.cc:482] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:15.735977 31420 raft_consensus.cc:504] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:15.736356 31420 leader_election.cc:288] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:15.736821 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b422c91ffe87402c91a0bc58a0715940" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:15.736900 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b422c91ffe87402c91a0bc58a0715940" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:15.736954 22632 raft_consensus.cc:2454] T b422c91ffe87402c91a0bc58a0715940 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:15.737033 22372 raft_consensus.cc:2454] T b422c91ffe87402c91a0bc58a0715940 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:15.737349 22439 leader_election.cc:302] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:15.738710 31420 raft_consensus.cc:2781] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:15.738775 31420 raft_consensus.cc:482] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:15.738816 31420 raft_consensus.cc:3037] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:15.739872 31420 raft_consensus.cc:504] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:15.740192 31420 leader_election.cc:288] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:15.740940 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b422c91ffe87402c91a0bc58a0715940" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:15.741055 22632 raft_consensus.cc:3037] T b422c91ffe87402c91a0bc58a0715940 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:15.742039 22632 raft_consensus.cc:2454] T b422c91ffe87402c91a0bc58a0715940 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:15.743703 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b422c91ffe87402c91a0bc58a0715940" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:15.743831 22372 raft_consensus.cc:3037] T b422c91ffe87402c91a0bc58a0715940 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:15.744771 22372 raft_consensus.cc:2454] T b422c91ffe87402c91a0bc58a0715940 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:15.745028 22439 leader_election.cc:302] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:15.745569 31420 raft_consensus.cc:2781] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:15.745657 31420 raft_consensus.cc:686] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:59:15.745795 31420 consensus_queue.cc:237] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:15.747999 22226 catalog_manager.cc:5484] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:15.802649 22500 raft_consensus.cc:1203] T a3ffa0ad72d7493082f697a1215bdbf4 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:15.826047 22222 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:15.831600 31768 tablet.cc:1722] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:15.832032 31769 tablet.cc:1722] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:15.832098 31771 tablet.cc:1722] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:15.837150 22229 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=d3d4077a11f2475ea6143fb721adca37] alter complete (version 3)
I20240502 07:59:15.861287 30954 raft_consensus.cc:482] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:15.861440 30954 raft_consensus.cc:504] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:15.861874 30954 leader_election.cc:288] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:15.862206 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b27345a5a0874524855f57f8903afe6c" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:15.862234 22500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b27345a5a0874524855f57f8903afe6c" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:15.862337 24821 raft_consensus.cc:2454] T b27345a5a0874524855f57f8903afe6c P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:15.862355 22500 raft_consensus.cc:2454] T b27345a5a0874524855f57f8903afe6c P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:15.862629 25150 leader_election.cc:302] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b; no voters:
I20240502 07:59:15.862805 30954 raft_consensus.cc:2781] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:15.862870 30954 raft_consensus.cc:482] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:15.862898 30954 raft_consensus.cc:3037] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:15.863840 30954 raft_consensus.cc:504] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:15.864217 30954 leader_election.cc:288] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:15.864533 22500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b27345a5a0874524855f57f8903afe6c" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:15.864665 22500 raft_consensus.cc:3037] T b27345a5a0874524855f57f8903afe6c P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:15.864709 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b27345a5a0874524855f57f8903afe6c" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:15.864830 24821 raft_consensus.cc:3037] T b27345a5a0874524855f57f8903afe6c P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:15.865504 22500 raft_consensus.cc:2454] T b27345a5a0874524855f57f8903afe6c P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:15.865667 24821 raft_consensus.cc:2454] T b27345a5a0874524855f57f8903afe6c P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:15.865798 25148 leader_election.cc:302] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:15.865962 30954 raft_consensus.cc:2781] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:15.866063 30954 raft_consensus.cc:686] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:15.866209 30954 consensus_queue.cc:237] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:15.869388 22229 catalog_manager.cc:5484] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:15.902814 25062 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:15.907419 25194 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:15.909313 24930 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:15.922591 24800 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:15.924369 22745 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:15.925254 22352 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:15.925473 22481 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:15.929497 22614 tablet_service.cc:1429] Tablet server has 22 leaders and 0 scanners
I20240502 07:59:15.939608 22229 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:15.939775 22229 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:15.950353 22229 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 3d5511d63ba84b6d948d303701f23f7f
I20240502 07:59:15.951186 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d5511d63ba84b6d948d303701f23f7f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:15 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:15.951486 22614 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d5511d63ba84b6d948d303701f23f7f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:15 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:15.951575 22352 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d5511d63ba84b6d948d303701f23f7f with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:15 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:15.951678 31786 tablet_replica.cc:330] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:15.951828 31786 raft_consensus.cc:2229] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:15.951944 31860 tablet_replica.cc:330] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:15.952061 31786 raft_consensus.cc:2258] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:15.952288 31860 raft_consensus.cc:2229] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:15.952453 31860 raft_consensus.cc:2258] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:15.952749 31786 ts_tablet_manager.cc:1875] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:15.953099 31860 ts_tablet_manager.cc:1875] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:15.955166 31786 ts_tablet_manager.cc:1888] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:15.955245 31786 log.cc:1196] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/3d5511d63ba84b6d948d303701f23f7f
I20240502 07:59:15.955838 31786 ts_tablet_manager.cc:1909] T 3d5511d63ba84b6d948d303701f23f7f P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:59:15.955917 31860 ts_tablet_manager.cc:1888] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:15.956063 31860 log.cc:1196] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/3d5511d63ba84b6d948d303701f23f7f
I20240502 07:59:15.956349 31860 ts_tablet_manager.cc:1909] T 3d5511d63ba84b6d948d303701f23f7f P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:59:15.956638 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 3d5511d63ba84b6d948d303701f23f7f (table rebalancer_dynamic_table_new_name [id=d3d4077a11f2475ea6143fb721adca37]) successfully deleted
I20240502 07:59:15.956992 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 3d5511d63ba84b6d948d303701f23f7f (table rebalancer_dynamic_table_new_name [id=d3d4077a11f2475ea6143fb721adca37]) successfully deleted
I20240502 07:59:15.966370 31861 tablet_replica.cc:330] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:15.966646 31861 raft_consensus.cc:2229] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:15.966825 31861 raft_consensus.cc:2258] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:15.967435 31861 ts_tablet_manager.cc:1875] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:15.971335 31861 ts_tablet_manager.cc:1888] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:15.971490 31861 log.cc:1196] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/3d5511d63ba84b6d948d303701f23f7f
I20240502 07:59:15.971849 31861 ts_tablet_manager.cc:1909] T 3d5511d63ba84b6d948d303701f23f7f P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:59:15.972635 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet 3d5511d63ba84b6d948d303701f23f7f (table rebalancer_dynamic_table_new_name [id=d3d4077a11f2475ea6143fb721adca37]) successfully deleted
I20240502 07:59:16.052706 22225 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:16.060078 22614 tablet_service.cc:1467] Processing CreateTablet for tablet 000d322210634cb48961d72425ecbfa5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=718612cdad874d5d9ece1ee45054e13c]), partition=
I20240502 07:59:16.060086 25062 tablet_service.cc:1467] Processing CreateTablet for tablet 000d322210634cb48961d72425ecbfa5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=718612cdad874d5d9ece1ee45054e13c]), partition=
I20240502 07:59:16.060369 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 000d322210634cb48961d72425ecbfa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:16.060372 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 000d322210634cb48961d72425ecbfa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:16.062345 31610 tablet_bootstrap.cc:492] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:16.062901 31752 tablet_bootstrap.cc:492] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:16.063895 31752 tablet_bootstrap.cc:654] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:16.063907 31610 tablet_bootstrap.cc:654] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:16.065111 24930 tablet_service.cc:1467] Processing CreateTablet for tablet 000d322210634cb48961d72425ecbfa5 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=718612cdad874d5d9ece1ee45054e13c]), partition=
I20240502 07:59:16.065397 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 000d322210634cb48961d72425ecbfa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:16.067641 31742 tablet_bootstrap.cc:492] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:16.068348 31742 tablet_bootstrap.cc:654] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:16.069451 31742 tablet_bootstrap.cc:492] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:16.069538 31742 ts_tablet_manager.cc:1367] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:16.070045 31742 raft_consensus.cc:348] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:16.070165 31742 raft_consensus.cc:374] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:16.070200 31742 raft_consensus.cc:729] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:16.070329 31742 consensus_queue.cc:260] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:16.070628 31742 ts_tablet_manager.cc:1398] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:16.073850 31752 tablet_bootstrap.cc:492] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:16.073947 31752 ts_tablet_manager.cc:1367] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 07:59:16.074445 31752 raft_consensus.cc:348] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:16.074551 31752 raft_consensus.cc:374] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:16.074582 31752 raft_consensus.cc:729] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:16.074695 31752 consensus_queue.cc:260] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:16.074973 31752 ts_tablet_manager.cc:1398] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:16.082022 31610 tablet_bootstrap.cc:492] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:16.082134 31610 ts_tablet_manager.cc:1367] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20240502 07:59:16.082682 31610 raft_consensus.cc:348] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:16.082788 31610 raft_consensus.cc:374] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:16.082818 31610 raft_consensus.cc:729] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:16.082935 31610 consensus_queue.cc:260] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:16.083252 31610 ts_tablet_manager.cc:1398] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:16.127705 22352 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:16.128253 24930 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:16.132602 30670 consensus_queue.cc:1035] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:16.137542 25194 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:16.148284 31420 consensus_queue.cc:1035] T b422c91ffe87402c91a0bc58a0715940 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:16.148509 22614 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 07:59:16.150291 25062 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:16.151325 22745 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:16.152982 22481 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:16.161589 24800 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:16.173135 31922 ts_tablet_manager.cc:902] T 423d6be23b044130b0c916f995dffced P 4b4e7eeb7f3f42119c0363a79febc54b: Initiating tablet copy from peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:16.173632 31922 tablet_copy_client.cc:308] T 423d6be23b044130b0c916f995dffced P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.129:38535
I20240502 07:59:16.174216 22392 tablet_copy_service.cc:140] P b533a7d8d97943609c6e28e3e36b78a4: Received BeginTabletCopySession request for tablet 423d6be23b044130b0c916f995dffced from peer 4b4e7eeb7f3f42119c0363a79febc54b ({username='slave'} at 127.15.129.133:34117)
I20240502 07:59:16.174305 22392 tablet_copy_service.cc:161] P b533a7d8d97943609c6e28e3e36b78a4: Beginning new tablet copy session on tablet 423d6be23b044130b0c916f995dffced from peer 4b4e7eeb7f3f42119c0363a79febc54b at {username='slave'} at 127.15.129.133:34117: session id = 4b4e7eeb7f3f42119c0363a79febc54b-423d6be23b044130b0c916f995dffced
I20240502 07:59:16.175215 22392 tablet_copy_source_session.cc:205] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:16.176164 31922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 423d6be23b044130b0c916f995dffced. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:16.178727 31922 tablet_copy_client.cc:783] T 423d6be23b044130b0c916f995dffced P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:16.178931 31922 tablet_copy_client.cc:647] T 423d6be23b044130b0c916f995dffced P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:16.200742 31589 consensus_queue.cc:1035] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:16.202348 31589 consensus_queue.cc:1035] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:16.203305 31922 tablet_copy_client.cc:523] T 423d6be23b044130b0c916f995dffced P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:16.205246 31922 tablet_bootstrap.cc:492] T 423d6be23b044130b0c916f995dffced P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:16.230222 31922 tablet_bootstrap.cc:492] T 423d6be23b044130b0c916f995dffced P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=635 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:16.230580 31922 tablet_bootstrap.cc:492] T 423d6be23b044130b0c916f995dffced P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap complete.
I20240502 07:59:16.230904 31922 ts_tablet_manager.cc:1367] T 423d6be23b044130b0c916f995dffced P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.026s user 0.025s sys 0.000s
I20240502 07:59:16.231361 31922 raft_consensus.cc:348] T 423d6be23b044130b0c916f995dffced P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:16.231500 31922 raft_consensus.cc:729] T 423d6be23b044130b0c916f995dffced P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:16.231654 31922 consensus_queue.cc:260] T 423d6be23b044130b0c916f995dffced P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:16.231942 31922 ts_tablet_manager.cc:1398] T 423d6be23b044130b0c916f995dffced P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:16.232632 22392 tablet_copy_service.cc:342] P b533a7d8d97943609c6e28e3e36b78a4: Request end of tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-423d6be23b044130b0c916f995dffced received from {username='slave'} at 127.15.129.133:34117
I20240502 07:59:16.232725 22392 tablet_copy_service.cc:433] P b533a7d8d97943609c6e28e3e36b78a4: ending tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-423d6be23b044130b0c916f995dffced on tablet 423d6be23b044130b0c916f995dffced with peer 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:16.236588 24952 consensus_queue.cc:237] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:16.237895 25215 raft_consensus.cc:1261] T a3ffa0ad72d7493082f697a1215bdbf4 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:16.238263 31589 consensus_queue.cc:1035] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:16.238377 22499 raft_consensus.cc:1261] T a3ffa0ad72d7493082f697a1215bdbf4 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:16.238692 31589 consensus_queue.cc:1035] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:16.240386 31586 raft_consensus.cc:2932] T a3ffa0ad72d7493082f697a1215bdbf4 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:16.240716 22499 raft_consensus.cc:2932] T a3ffa0ad72d7493082f697a1215bdbf4 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:16.241899 25215 raft_consensus.cc:2932] T a3ffa0ad72d7493082f697a1215bdbf4 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:16.244686 22226 catalog_manager.cc:5484] T a3ffa0ad72d7493082f697a1215bdbf4 P fabddb16c85944648a710ec52fc7244c reported cstate change: config changed from index 2 to 3, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) evicted. New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:16.247846 24800 tablet_service.cc:1514] Processing DeleteTablet for tablet a3ffa0ad72d7493082f697a1215bdbf4 with delete_type TABLET_DATA_TOMBSTONED (TS 4b4e7eeb7f3f42119c0363a79febc54b not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:36878
I20240502 07:59:16.249356 31951 tablet_replica.cc:330] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b: stopping tablet replica
I20240502 07:59:16.249549 31951 raft_consensus.cc:2229] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:16.249724 31951 raft_consensus.cc:2258] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:16.250226 31951 ts_tablet_manager.cc:1875] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:16.252630 31951 ts_tablet_manager.cc:1888] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:16.252754 31951 log.cc:1196] T a3ffa0ad72d7493082f697a1215bdbf4 P 4b4e7eeb7f3f42119c0363a79febc54b: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/wals/a3ffa0ad72d7493082f697a1215bdbf4
I20240502 07:59:16.253377 22216 catalog_manager.cc:4830] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): tablet a3ffa0ad72d7493082f697a1215bdbf4 (table rebalancer_extra_table_0 [id=ad840b2a5e7149b78acd24dc98f05685]) successfully deleted
I20240502 07:59:16.253939 31950 raft_consensus.cc:482] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:16.254025 31950 raft_consensus.cc:504] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:16.254688 31950 leader_election.cc:288] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:16.254695 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "000d322210634cb48961d72425ecbfa5" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:16.254760 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "000d322210634cb48961d72425ecbfa5" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:16.254819 25083 raft_consensus.cc:2454] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:16.254876 22634 raft_consensus.cc:2454] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:16.255146 24888 leader_election.cc:302] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:16.255337 31950 raft_consensus.cc:2781] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:16.255396 31950 raft_consensus.cc:482] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:16.255431 31950 raft_consensus.cc:3037] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:16.256402 31950 raft_consensus.cc:504] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:16.256918 31950 leader_election.cc:288] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:16.256984 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "000d322210634cb48961d72425ecbfa5" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:16.257124 25083 raft_consensus.cc:3037] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:16.257138 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "000d322210634cb48961d72425ecbfa5" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:16.257249 22634 raft_consensus.cc:3037] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:16.258075 25083 raft_consensus.cc:2454] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:16.258159 22634 raft_consensus.cc:2454] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:16.258447 24888 leader_election.cc:302] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:16.258615 31586 raft_consensus.cc:2781] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:16.258697 31586 raft_consensus.cc:686] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:16.258826 31586 consensus_queue.cc:237] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:16.262159 22221 catalog_manager.cc:5484] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:16.279233 30954 consensus_queue.cc:1035] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:16.280973 22634 consensus_queue.cc:237] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:16.283015 22501 raft_consensus.cc:1261] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20240502 07:59:16.283327 22764 raft_consensus.cc:1261] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20240502 07:59:16.283445 22569 consensus_peers.cc:485] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d61cb2b6dce1409bb30d1c68c970dc22. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:16.283560 31717 consensus_queue.cc:1035] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:16.283816 31410 consensus_queue.cc:1035] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:16.284551 31325 consensus_queue.cc:1035] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:16.288597 31717 raft_consensus.cc:2932] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:16.289088 22764 raft_consensus.cc:2932] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:16.291244 22499 raft_consensus.cc:2932] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:16.293931 22221 catalog_manager.cc:5484] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index -1 to 4, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:16.375864 22229 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:16.380239 22632 raft_consensus.cc:1261] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:16.380590 25084 raft_consensus.cc:1261] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:16.380651 31950 consensus_queue.cc:1035] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:16.381065 31950 consensus_queue.cc:1035] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:16.383397 31929 mvcc.cc:205] Tried to move back new op lower bound from 7023152154128896000 to 7023152153637638144. Current Snapshot: MvccSnapshot[applied={T|T < 7023152154128896000}]
I20240502 07:59:16.383728 32023 tablet.cc:1722] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:16.384137 32023 tablet.cc:1620] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
I20240502 07:59:16.385761 22224 catalog_manager.cc:5576] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733) does not have the latest schema for tablet 000d322210634cb48961d72425ecbfa5 (table rebalancer_dynamic_table [id=718612cdad874d5d9ece1ee45054e13c]). Expected version 1 got 0
I20240502 07:59:16.386608 22228 catalog_manager.cc:5576] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703) does not have the latest schema for tablet 000d322210634cb48961d72425ecbfa5 (table rebalancer_dynamic_table [id=718612cdad874d5d9ece1ee45054e13c]). Expected version 1 got 0
I20240502 07:59:16.389048 32022 tablet.cc:1722] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:16.389442 32022 tablet.cc:1620] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:16.390347 22228 catalog_manager.cc:5576] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733) does not have the latest schema for tablet 000d322210634cb48961d72425ecbfa5 (table rebalancer_dynamic_table [id=718612cdad874d5d9ece1ee45054e13c]). Expected version 1 got 0
I20240502 07:59:16.395390 32025 tablet.cc:1722] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:16.396222 32025 tablet.cc:1620] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:16.398036 22228 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=718612cdad874d5d9ece1ee45054e13c] alter complete (version 1)
I20240502 07:59:16.411990 24930 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 07:59:16.418678 25194 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:16.424906 24800 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:16.425199 25062 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:16.441224 22228 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:16.442750 22481 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:16.446856 22352 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:16.450086 25194 tablet_service.cc:1467] Processing CreateTablet for tablet 423624773b664098ad16c43dd7454e47 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=1e844a21753d401ab68415fb8bb54f67]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:16.450371 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 423624773b664098ad16c43dd7454e47 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=1e844a21753d401ab68415fb8bb54f67]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:16.450470 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 423624773b664098ad16c43dd7454e47. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:16.450656 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 423624773b664098ad16c43dd7454e47. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:16.453071 25193 tablet_service.cc:1467] Processing CreateTablet for tablet f827f414e7cc4ede8e41716b6e2efcf9 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=1e844a21753d401ab68415fb8bb54f67]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:16.453135 22743 tablet_service.cc:1467] Processing CreateTablet for tablet e53e59fe26f54ea4bc2f946efea2ae7f (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=1e844a21753d401ab68415fb8bb54f67]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:16.453369 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f827f414e7cc4ede8e41716b6e2efcf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:16.453418 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e53e59fe26f54ea4bc2f946efea2ae7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:16.455657 22614 tablet_service.cc:1467] Processing CreateTablet for tablet e53e59fe26f54ea4bc2f946efea2ae7f (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=1e844a21753d401ab68415fb8bb54f67]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:16.455675 22352 tablet_service.cc:1467] Processing CreateTablet for tablet f827f414e7cc4ede8e41716b6e2efcf9 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=1e844a21753d401ab68415fb8bb54f67]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:16.455932 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e53e59fe26f54ea4bc2f946efea2ae7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:16.455937 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f827f414e7cc4ede8e41716b6e2efcf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:16.458456 31610 tablet_bootstrap.cc:492] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:16.459429 31610 tablet_bootstrap.cc:654] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:16.463390 24800 tablet_service.cc:1467] Processing CreateTablet for tablet f827f414e7cc4ede8e41716b6e2efcf9 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=1e844a21753d401ab68415fb8bb54f67]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:16.463682 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f827f414e7cc4ede8e41716b6e2efcf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:16.464918 25062 tablet_service.cc:1467] Processing CreateTablet for tablet e53e59fe26f54ea4bc2f946efea2ae7f (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=1e844a21753d401ab68415fb8bb54f67]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:16.465186 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e53e59fe26f54ea4bc2f946efea2ae7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:16.465812 22481 tablet_service.cc:1467] Processing CreateTablet for tablet 423624773b664098ad16c43dd7454e47 (DEFAULT_TABLE table=rebalancer_extra_table_50 [id=1e844a21753d401ab68415fb8bb54f67]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:16.466101 22481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 423624773b664098ad16c43dd7454e47. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:16.467500 31752 tablet_bootstrap.cc:492] T e53e59fe26f54ea4bc2f946efea2ae7f P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:16.468545 31752 tablet_bootstrap.cc:654] T e53e59fe26f54ea4bc2f946efea2ae7f P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:16.476032 32039 tablet_bootstrap.cc:492] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:16.476285 22614 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 07:59:16.477310 32039 tablet_bootstrap.cc:654] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:16.480058 32037 tablet_bootstrap.cc:492] T 423624773b664098ad16c43dd7454e47 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:16.484918 32038 tablet_bootstrap.cc:492] T 423624773b664098ad16c43dd7454e47 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:16.487273 31752 tablet_bootstrap.cc:492] T e53e59fe26f54ea4bc2f946efea2ae7f P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:16.487380 31752 ts_tablet_manager.cc:1367] T e53e59fe26f54ea4bc2f946efea2ae7f P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240502 07:59:16.487862 31752 raft_consensus.cc:348] T e53e59fe26f54ea4bc2f946efea2ae7f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:16.487977 31752 raft_consensus.cc:374] T e53e59fe26f54ea4bc2f946efea2ae7f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:16.488004 31752 raft_consensus.cc:729] T e53e59fe26f54ea4bc2f946efea2ae7f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:16.488114 31752 consensus_queue.cc:260] T e53e59fe26f54ea4bc2f946efea2ae7f P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:16.488382 31752 ts_tablet_manager.cc:1398] T e53e59fe26f54ea4bc2f946efea2ae7f P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:16.496354 31610 tablet_bootstrap.cc:492] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:16.496467 31610 ts_tablet_manager.cc:1367] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.000s
I20240502 07:59:16.496937 31610 raft_consensus.cc:348] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:16.497045 31610 raft_consensus.cc:374] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:16.497071 31610 raft_consensus.cc:729] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:16.497182 31610 consensus_queue.cc:260] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:16.497462 31610 ts_tablet_manager.cc:1398] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:16.497888 32037 tablet_bootstrap.cc:654] T 423624773b664098ad16c43dd7454e47 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:16.498991 32037 tablet_bootstrap.cc:492] T 423624773b664098ad16c43dd7454e47 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:16.501286 32037 ts_tablet_manager.cc:1367] T 423624773b664098ad16c43dd7454e47 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 07:59:16.501967 32041 tablet_bootstrap.cc:492] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:16.503688 32037 raft_consensus.cc:348] T 423624773b664098ad16c43dd7454e47 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:16.504413 22228 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:16.504642 32037 raft_consensus.cc:374] T 423624773b664098ad16c43dd7454e47 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:16.504756 32037 raft_consensus.cc:729] T 423624773b664098ad16c43dd7454e47 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:16.505452 32037 consensus_queue.cc:260] T 423624773b664098ad16c43dd7454e47 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:16.505913 32037 ts_tablet_manager.cc:1398] T 423624773b664098ad16c43dd7454e47 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:16.506044 32039 tablet_bootstrap.cc:492] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:16.506188 32039 ts_tablet_manager.cc:1367] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20240502 07:59:16.506669 32039 raft_consensus.cc:348] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:16.506834 32039 raft_consensus.cc:374] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:16.506911 32039 raft_consensus.cc:729] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:16.507088 32037 tablet_bootstrap.cc:492] T f827f414e7cc4ede8e41716b6e2efcf9 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:16.507387 32039 consensus_queue.cc:260] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:16.507750 32039 ts_tablet_manager.cc:1398] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:16.508344 32041 tablet_bootstrap.cc:654] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:16.509356 32037 tablet_bootstrap.cc:654] T f827f414e7cc4ede8e41716b6e2efcf9 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:16.515480 32022 tablet.cc:1722] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:16.515822 32022 tablet.cc:1620] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:16.517760 32038 tablet_bootstrap.cc:654] T 423624773b664098ad16c43dd7454e47 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:16.517949 32023 tablet.cc:1722] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:16.518277 32023 tablet.cc:1620] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
I20240502 07:59:16.518388 22743 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:16.524811 22224 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=718612cdad874d5d9ece1ee45054e13c] alter complete (version 2)
I20240502 07:59:16.525674 32025 tablet.cc:1722] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:16.526053 32025 tablet.cc:1620] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:16.527855 32038 tablet_bootstrap.cc:492] T 423624773b664098ad16c43dd7454e47 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:16.528015 32038 ts_tablet_manager.cc:1367] T 423624773b664098ad16c43dd7454e47 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.044s user 0.003s sys 0.000s
I20240502 07:59:16.528574 32038 raft_consensus.cc:348] T 423624773b664098ad16c43dd7454e47 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:16.528762 32038 raft_consensus.cc:374] T 423624773b664098ad16c43dd7454e47 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:16.528829 32038 raft_consensus.cc:729] T 423624773b664098ad16c43dd7454e47 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:16.529453 32038 consensus_queue.cc:260] T 423624773b664098ad16c43dd7454e47 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:16.530573 32038 ts_tablet_manager.cc:1398] T 423624773b664098ad16c43dd7454e47 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:16.531033 32038 tablet_bootstrap.cc:492] T e53e59fe26f54ea4bc2f946efea2ae7f P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:16.533486 32038 tablet_bootstrap.cc:654] T e53e59fe26f54ea4bc2f946efea2ae7f P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:16.535511 31755 raft_consensus.cc:482] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:16.535626 31755 raft_consensus.cc:504] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:16.535877 32037 tablet_bootstrap.cc:492] T f827f414e7cc4ede8e41716b6e2efcf9 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:16.535965 32037 ts_tablet_manager.cc:1367] T f827f414e7cc4ede8e41716b6e2efcf9 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.029s user 0.000s sys 0.001s
I20240502 07:59:16.536017 31755 leader_election.cc:288] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:16.536410 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f827f414e7cc4ede8e41716b6e2efcf9" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:16.536399 32037 raft_consensus.cc:348] T f827f414e7cc4ede8e41716b6e2efcf9 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:16.536509 32037 raft_consensus.cc:374] T f827f414e7cc4ede8e41716b6e2efcf9 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:16.536509 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f827f414e7cc4ede8e41716b6e2efcf9" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
W20240502 07:59:16.536796 22307 leader_election.cc:341] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Illegal state: must be running to vote when last-logged opid is not known
I20240502 07:59:16.536551 32037 raft_consensus.cc:729] T f827f414e7cc4ede8e41716b6e2efcf9 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:16.537207 32037 consensus_queue.cc:260] T f827f414e7cc4ede8e41716b6e2efcf9 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:16.537537 32037 ts_tablet_manager.cc:1398] T f827f414e7cc4ede8e41716b6e2efcf9 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:16.538743 25215 raft_consensus.cc:2454] T f827f414e7cc4ede8e41716b6e2efcf9 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:16.539745 22308 leader_election.cc:302] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, b533a7d8d97943609c6e28e3e36b78a4; no voters: 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:16.539902 31755 raft_consensus.cc:2781] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:16.539952 31755 raft_consensus.cc:482] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:16.539997 31755 raft_consensus.cc:3037] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:16.541196 31755 raft_consensus.cc:504] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:16.541666 31755 leader_election.cc:288] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:16.542086 24821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f827f414e7cc4ede8e41716b6e2efcf9" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
W20240502 07:59:16.542515 22307 leader_election.cc:341] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Illegal state: must be running to vote when last-logged opid is not known
I20240502 07:59:16.542903 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f827f414e7cc4ede8e41716b6e2efcf9" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:16.543020 25215 raft_consensus.cc:3037] T f827f414e7cc4ede8e41716b6e2efcf9 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:16.544163 25215 raft_consensus.cc:2454] T f827f414e7cc4ede8e41716b6e2efcf9 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:16.544533 22308 leader_election.cc:302] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, b533a7d8d97943609c6e28e3e36b78a4; no voters: 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:16.544744 31755 raft_consensus.cc:2781] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:16.544945 31755 raft_consensus.cc:686] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:59:16.545105 31755 consensus_queue.cc:237] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:16.547917 22226 catalog_manager.cc:5484] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:16.513613 32042 tablet_bootstrap.cc:492] T f827f414e7cc4ede8e41716b6e2efcf9 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:16.552527 32042 tablet_bootstrap.cc:654] T f827f414e7cc4ede8e41716b6e2efcf9 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:16.554008 32041 tablet_bootstrap.cc:492] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:16.554116 32041 ts_tablet_manager.cc:1367] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.052s user 0.002s sys 0.000s
I20240502 07:59:16.554621 32041 raft_consensus.cc:348] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:16.554754 32041 raft_consensus.cc:374] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:16.554791 32041 raft_consensus.cc:729] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:16.554930 32041 consensus_queue.cc:260] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:16.556259 32041 ts_tablet_manager.cc:1398] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:16.563014 32042 tablet_bootstrap.cc:492] T f827f414e7cc4ede8e41716b6e2efcf9 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:16.563122 32042 ts_tablet_manager.cc:1367] T f827f414e7cc4ede8e41716b6e2efcf9 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.050s user 0.002s sys 0.000s
I20240502 07:59:16.563273 32038 tablet_bootstrap.cc:492] T e53e59fe26f54ea4bc2f946efea2ae7f P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:16.563385 32038 ts_tablet_manager.cc:1367] T e53e59fe26f54ea4bc2f946efea2ae7f P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20240502 07:59:16.563829 32038 raft_consensus.cc:348] T e53e59fe26f54ea4bc2f946efea2ae7f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:16.563951 32038 raft_consensus.cc:374] T e53e59fe26f54ea4bc2f946efea2ae7f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:16.563884 32042 raft_consensus.cc:348] T f827f414e7cc4ede8e41716b6e2efcf9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:16.563987 32038 raft_consensus.cc:729] T e53e59fe26f54ea4bc2f946efea2ae7f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:16.564000 32042 raft_consensus.cc:374] T f827f414e7cc4ede8e41716b6e2efcf9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:16.564033 32042 raft_consensus.cc:729] T f827f414e7cc4ede8e41716b6e2efcf9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:16.564111 32038 consensus_queue.cc:260] T e53e59fe26f54ea4bc2f946efea2ae7f P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:16.564152 32042 consensus_queue.cc:260] T f827f414e7cc4ede8e41716b6e2efcf9 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:16.564410 32038 ts_tablet_manager.cc:1398] T e53e59fe26f54ea4bc2f946efea2ae7f P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:16.565439 32042 ts_tablet_manager.cc:1398] T f827f414e7cc4ede8e41716b6e2efcf9 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:16.626827 22226 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:16.636304 32022 tablet.cc:1722] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:16.637780 32025 tablet.cc:1722] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:16.637984 32023 tablet.cc:1722] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:16.648097 22228 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=718612cdad874d5d9ece1ee45054e13c] alter complete (version 3)
I20240502 07:59:16.750527 22229 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:16.750725 22229 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:16.753082 22229 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 000d322210634cb48961d72425ecbfa5
I20240502 07:59:16.753715 25062 tablet_service.cc:1514] Processing DeleteTablet for tablet 000d322210634cb48961d72425ecbfa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:16 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:16.753847 24930 tablet_service.cc:1514] Processing DeleteTablet for tablet 000d322210634cb48961d72425ecbfa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:16 UTC) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:16.754027 32089 tablet_replica.cc:330] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:16.754616 32089 raft_consensus.cc:2229] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:16.754694 22614 tablet_service.cc:1514] Processing DeleteTablet for tablet 000d322210634cb48961d72425ecbfa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:16 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:16.754791 32089 raft_consensus.cc:2258] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:16.754849 32090 tablet_replica.cc:330] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:16.754994 32090 raft_consensus.cc:2229] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:16.755039 32091 tablet_replica.cc:330] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:16.755190 32091 raft_consensus.cc:2229] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:16.755242 32090 raft_consensus.cc:2258] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:16.755362 32091 raft_consensus.cc:2258] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:16.755476 32089 ts_tablet_manager.cc:1875] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:16.755911 32090 ts_tablet_manager.cc:1875] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:16.756023 32091 ts_tablet_manager.cc:1875] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:16.757498 32089 ts_tablet_manager.cc:1888] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:16.757607 32089 log.cc:1196] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/000d322210634cb48961d72425ecbfa5
I20240502 07:59:16.757910 32089 ts_tablet_manager.cc:1909] T 000d322210634cb48961d72425ecbfa5 P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:59:16.758311 32091 ts_tablet_manager.cc:1888] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:16.758421 32091 log.cc:1196] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/000d322210634cb48961d72425ecbfa5
I20240502 07:59:16.758549 32090 ts_tablet_manager.cc:1888] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:16.758637 32090 log.cc:1196] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/000d322210634cb48961d72425ecbfa5
I20240502 07:59:16.758693 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet 000d322210634cb48961d72425ecbfa5 (table rebalancer_dynamic_table_new_name [id=718612cdad874d5d9ece1ee45054e13c]) successfully deleted
I20240502 07:59:16.758783 32091 ts_tablet_manager.cc:1909] T 000d322210634cb48961d72425ecbfa5 P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:59:16.759402 32090 ts_tablet_manager.cc:1909] T 000d322210634cb48961d72425ecbfa5 P 1aecef81733c47b3b468504934357231: Deleting consensus metadata
I20240502 07:59:16.759508 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 000d322210634cb48961d72425ecbfa5 (table rebalancer_dynamic_table_new_name [id=718612cdad874d5d9ece1ee45054e13c]) successfully deleted
I20240502 07:59:16.760004 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet 000d322210634cb48961d72425ecbfa5 (table rebalancer_dynamic_table_new_name [id=718612cdad874d5d9ece1ee45054e13c]) successfully deleted
I20240502 07:59:16.776891 24821 raft_consensus.cc:1203] T 423d6be23b044130b0c916f995dffced P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:16.802935 31535 raft_consensus.cc:482] T f827f414e7cc4ede8e41716b6e2efcf9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:16.803061 31535 raft_consensus.cc:504] T f827f414e7cc4ede8e41716b6e2efcf9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:16.803977 31535 leader_election.cc:288] T f827f414e7cc4ede8e41716b6e2efcf9 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:16.804668 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f827f414e7cc4ede8e41716b6e2efcf9" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:16.804929 25215 raft_consensus.cc:2379] T f827f414e7cc4ede8e41716b6e2efcf9 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4b4e7eeb7f3f42119c0363a79febc54b in current term 1: Already voted for candidate b533a7d8d97943609c6e28e3e36b78a4 in this term.
I20240502 07:59:16.805217 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f827f414e7cc4ede8e41716b6e2efcf9" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:16.805601 24757 leader_election.cc:302] T f827f414e7cc4ede8e41716b6e2efcf9 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b; no voters: 43930dee0cd5498fa8e357a34dd71432, b533a7d8d97943609c6e28e3e36b78a4
I20240502 07:59:16.805826 31535 raft_consensus.cc:3037] T f827f414e7cc4ede8e41716b6e2efcf9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:16.806782 31535 raft_consensus.cc:2726] T f827f414e7cc4ede8e41716b6e2efcf9 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 07:59:16.827821 32092 ts_tablet_manager.cc:902] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:16.828284 32092 tablet_copy_client.cc:308] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:59:16.828825 22653 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet d61cb2b6dce1409bb30d1c68c970dc22 from peer 4b4e7eeb7f3f42119c0363a79febc54b ({username='slave'} at 127.15.129.133:55755)
I20240502 07:59:16.828933 22653 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet d61cb2b6dce1409bb30d1c68c970dc22 from peer 4b4e7eeb7f3f42119c0363a79febc54b at {username='slave'} at 127.15.129.133:55755: session id = 4b4e7eeb7f3f42119c0363a79febc54b-d61cb2b6dce1409bb30d1c68c970dc22
I20240502 07:59:16.829770 22653 tablet_copy_source_session.cc:205] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:16.830621 32092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d61cb2b6dce1409bb30d1c68c970dc22. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:16.832456 32092 tablet_copy_client.cc:783] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:16.832600 32092 tablet_copy_client.cc:647] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:16.833767 32092 tablet_copy_client.cc:523] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:16.835352 32092 tablet_bootstrap.cc:492] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:16.855643 22230 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:16.856484 32092 tablet_bootstrap.cc:492] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:16.856781 32092 tablet_bootstrap.cc:492] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap complete.
I20240502 07:59:16.856977 32092 ts_tablet_manager.cc:1367] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.022s user 0.020s sys 0.000s
I20240502 07:59:16.857738 32092 raft_consensus.cc:348] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:16.857882 32092 raft_consensus.cc:729] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:16.858044 32092 consensus_queue.cc:260] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:16.858181 31717 raft_consensus.cc:482] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:16.858278 31717 raft_consensus.cc:504] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:16.858357 32092 ts_tablet_manager.cc:1398] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:16.858669 31717 leader_election.cc:288] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:16.858836 22653 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-d61cb2b6dce1409bb30d1c68c970dc22 received from {username='slave'} at 127.15.129.133:55755
I20240502 07:59:16.858927 22653 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-d61cb2b6dce1409bb30d1c68c970dc22 on tablet d61cb2b6dce1409bb30d1c68c970dc22 with peer 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:16.859220 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e53e59fe26f54ea4bc2f946efea2ae7f" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:16.859362 22764 raft_consensus.cc:2454] T e53e59fe26f54ea4bc2f946efea2ae7f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:16.859747 22570 leader_election.cc:302] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:16.860127 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e53e59fe26f54ea4bc2f946efea2ae7f" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:16.860749 25084 raft_consensus.cc:2454] T e53e59fe26f54ea4bc2f946efea2ae7f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:16.861941 31717 raft_consensus.cc:2781] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:16.862011 31717 raft_consensus.cc:482] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:16.862051 31717 raft_consensus.cc:3037] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:16.863003 31717 raft_consensus.cc:504] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:16.863394 31717 leader_election.cc:288] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:16.863420 22481 tablet_service.cc:1467] Processing CreateTablet for tablet c89b62ba3dde4b9c9aeebbd031572030 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2104010124fd4c5cba27783dd01ff099]), partition=
I20240502 07:59:16.863667 22481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c89b62ba3dde4b9c9aeebbd031572030. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:16.863669 22614 tablet_service.cc:1467] Processing CreateTablet for tablet c89b62ba3dde4b9c9aeebbd031572030 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2104010124fd4c5cba27783dd01ff099]), partition=
I20240502 07:59:16.863847 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c89b62ba3dde4b9c9aeebbd031572030. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:16.865624 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e53e59fe26f54ea4bc2f946efea2ae7f" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:16.865754 22764 raft_consensus.cc:3037] T e53e59fe26f54ea4bc2f946efea2ae7f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:16.865756 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e53e59fe26f54ea4bc2f946efea2ae7f" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:16.865859 32041 tablet_bootstrap.cc:492] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:16.865864 25084 raft_consensus.cc:3037] T e53e59fe26f54ea4bc2f946efea2ae7f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:16.865965 22743 tablet_service.cc:1467] Processing CreateTablet for tablet c89b62ba3dde4b9c9aeebbd031572030 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=2104010124fd4c5cba27783dd01ff099]), partition=
I20240502 07:59:16.866168 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c89b62ba3dde4b9c9aeebbd031572030. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:16.866778 25084 raft_consensus.cc:2454] T e53e59fe26f54ea4bc2f946efea2ae7f P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:16.866777 22764 raft_consensus.cc:2454] T e53e59fe26f54ea4bc2f946efea2ae7f P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:16.867292 32041 tablet_bootstrap.cc:654] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:16.867935 22567 leader_election.cc:302] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:16.868348 32038 tablet_bootstrap.cc:492] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:16.868359 31410 raft_consensus.cc:2781] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:16.868410 31610 tablet_bootstrap.cc:492] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:16.868492 31410 raft_consensus.cc:686] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:59:16.868618 31410 consensus_queue.cc:237] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:16.869257 32038 tablet_bootstrap.cc:654] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:16.869288 31610 tablet_bootstrap.cc:654] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:16.870999 32038 tablet_bootstrap.cc:492] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:16.871099 32038 ts_tablet_manager.cc:1367] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:16.871574 32038 raft_consensus.cc:348] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:16.871680 32038 raft_consensus.cc:374] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:16.871714 32038 raft_consensus.cc:729] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:16.871824 32038 consensus_queue.cc:260] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:16.872128 32038 ts_tablet_manager.cc:1398] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:16.872336 31610 tablet_bootstrap.cc:492] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:16.872423 31610 ts_tablet_manager.cc:1367] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240502 07:59:16.872429 32041 tablet_bootstrap.cc:492] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:16.872505 32041 ts_tablet_manager.cc:1367] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:16.872886 31610 raft_consensus.cc:348] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:16.872921 32041 raft_consensus.cc:348] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:16.872998 31610 raft_consensus.cc:374] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:16.873011 32041 raft_consensus.cc:374] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:16.873034 31610 raft_consensus.cc:729] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:16.873044 32041 raft_consensus.cc:729] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:16.873148 31610 consensus_queue.cc:260] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:16.873159 32041 consensus_queue.cc:260] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:16.873448 31610 ts_tablet_manager.cc:1398] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:16.873448 32041 ts_tablet_manager.cc:1398] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:16.874158 22230 catalog_manager.cc:5484] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:16.885558 31770 raft_consensus.cc:482] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:16.885685 31770 raft_consensus.cc:504] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:16.886119 31770 leader_election.cc:288] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:16.886440 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "423624773b664098ad16c43dd7454e47" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:16.886480 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "423624773b664098ad16c43dd7454e47" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:16.886579 25215 raft_consensus.cc:2454] T 423624773b664098ad16c43dd7454e47 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:16.886626 22764 raft_consensus.cc:2454] T 423624773b664098ad16c43dd7454e47 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:16.886864 22439 leader_election.cc:302] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:16.887133 31770 raft_consensus.cc:2781] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:16.887182 31770 raft_consensus.cc:482] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:16.887216 31770 raft_consensus.cc:3037] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:16.888208 31770 raft_consensus.cc:504] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:16.888578 31770 leader_election.cc:288] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:16.888957 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "423624773b664098ad16c43dd7454e47" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:16.888947 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "423624773b664098ad16c43dd7454e47" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:16.889061 25215 raft_consensus.cc:3037] T 423624773b664098ad16c43dd7454e47 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:16.889102 22764 raft_consensus.cc:3037] T 423624773b664098ad16c43dd7454e47 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:16.890044 25215 raft_consensus.cc:2454] T 423624773b664098ad16c43dd7454e47 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:16.890044 22764 raft_consensus.cc:2454] T 423624773b664098ad16c43dd7454e47 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:16.890317 22439 leader_election.cc:302] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:16.890655 31770 raft_consensus.cc:2781] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:16.890735 31770 raft_consensus.cc:686] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:59:16.890867 31770 consensus_queue.cc:237] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:16.893131 22229 catalog_manager.cc:5484] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:16.918203 24930 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:16.924444 25062 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:16.924870 24800 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:16.939988 25193 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:16.941428 22481 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:16.942560 32049 consensus_queue.cc:1035] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:16.943744 22352 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:16.956172 22614 tablet_service.cc:1429] Tablet server has 22 leaders and 0 scanners
I20240502 07:59:16.961859 32049 consensus_queue.cc:1035] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:16.965029 22743 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:17.031705 31905 raft_consensus.cc:482] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:17.031862 31905 raft_consensus.cc:504] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.032454 31905 leader_election.cc:288] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:17.032801 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89b62ba3dde4b9c9aeebbd031572030" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:17.032938 22632 raft_consensus.cc:2454] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:17.032969 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89b62ba3dde4b9c9aeebbd031572030" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:17.033113 22501 raft_consensus.cc:2454] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:17.033457 22698 leader_election.cc:302] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:17.033826 31905 raft_consensus.cc:2781] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:17.033887 31905 raft_consensus.cc:482] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:17.033933 31905 raft_consensus.cc:3037] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:17.035138 31905 raft_consensus.cc:504] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.035542 31905 leader_election.cc:288] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:17.035960 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89b62ba3dde4b9c9aeebbd031572030" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:17.036083 22501 raft_consensus.cc:3037] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:17.037043 22501 raft_consensus.cc:2454] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:17.037365 22698 leader_election.cc:302] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:17.037707 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89b62ba3dde4b9c9aeebbd031572030" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:17.037824 22632 raft_consensus.cc:3037] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:17.037660 31905 raft_consensus.cc:2781] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:17.037997 31905 raft_consensus.cc:686] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:59:17.038169 31905 consensus_queue.cc:237] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.038777 22632 raft_consensus.cc:2454] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:17.040686 22229 catalog_manager.cc:5484] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:17.140077 25193 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:17.144907 24800 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:17.145155 24930 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:17.146057 22614 tablet_service.cc:1429] Tablet server has 22 leaders and 0 scanners
I20240502 07:59:17.148298 25062 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:17.155972 22352 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:17.172564 22743 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:17.172580 22481 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:17.174757 22224 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:17.202674 22501 raft_consensus.cc:1261] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:17.203043 22632 raft_consensus.cc:1261] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:17.203071 31905 consensus_queue.cc:1035] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:17.203886 31905 consensus_queue.cc:1035] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:17.208325 22224 catalog_manager.cc:5576] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703) does not have the latest schema for tablet c89b62ba3dde4b9c9aeebbd031572030 (table rebalancer_dynamic_table [id=2104010124fd4c5cba27783dd01ff099]). Expected version 1 got 0
I20240502 07:59:17.210546 22224 catalog_manager.cc:5576] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625) does not have the latest schema for tablet c89b62ba3dde4b9c9aeebbd031572030 (table rebalancer_dynamic_table [id=2104010124fd4c5cba27783dd01ff099]). Expected version 1 got 0
I20240502 07:59:17.212991 22224 catalog_manager.cc:5576] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625) does not have the latest schema for tablet c89b62ba3dde4b9c9aeebbd031572030 (table rebalancer_dynamic_table [id=2104010124fd4c5cba27783dd01ff099]). Expected version 1 got 0
I20240502 07:59:17.216642 32185 tablet.cc:1722] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:17.216969 32185 tablet.cc:1620] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c: MemRowSet was empty: no flush needed.
I20240502 07:59:17.221668 22224 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=2104010124fd4c5cba27783dd01ff099] alter complete (version 1)
I20240502 07:59:17.223156 32182 tablet.cc:1722] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:17.223505 32182 tablet.cc:1620] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:59:17.224342 32186 tablet.cc:1722] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:17.224670 32186 tablet.cc:1620] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
I20240502 07:59:17.258951 22370 consensus_queue.cc:237] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:17.260306 24821 raft_consensus.cc:1261] T 423d6be23b044130b0c916f995dffced P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:17.260773 22764 raft_consensus.cc:1261] T 423d6be23b044130b0c916f995dffced P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20240502 07:59:17.260795 32049 consensus_queue.cc:1035] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:17.261178 32049 consensus_queue.cc:1035] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:17.262869 32049 raft_consensus.cc:2932] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:17.263460 22764 raft_consensus.cc:2932] T 423d6be23b044130b0c916f995dffced P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:17.265013 24820 raft_consensus.cc:2932] T 423d6be23b044130b0c916f995dffced P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.5: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:17.266920 22224 catalog_manager.cc:5484] T 423d6be23b044130b0c916f995dffced P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index 4 to 5, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:17.269831 22614 tablet_service.cc:1514] Processing DeleteTablet for tablet 423d6be23b044130b0c916f995dffced with delete_type TABLET_DATA_TOMBSTONED (TS e51b10af49484aaa8f0267aebc1bf27f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:17.272383 32206 tablet_replica.cc:330] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:17.272557 32206 raft_consensus.cc:2229] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:17.272724 32206 raft_consensus.cc:2258] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:17.273195 32206 ts_tablet_manager.cc:1875] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:17.275586 32206 ts_tablet_manager.cc:1888] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.4
I20240502 07:59:17.275715 32206 log.cc:1196] T 423d6be23b044130b0c916f995dffced P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/423d6be23b044130b0c916f995dffced
I20240502 07:59:17.276786 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 423d6be23b044130b0c916f995dffced (table rebalance_test_table_11 [id=423ebbc6961345159cc3d4c9668fe1dd]) successfully deleted
I20240502 07:59:17.315603 31717 consensus_queue.cc:1035] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:17.319792 32099 consensus_queue.cc:1035] T e53e59fe26f54ea4bc2f946efea2ae7f P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:17.323743 31770 consensus_queue.cc:1035] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:17.327702 22229 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:17.330473 31420 consensus_queue.cc:1035] T 423624773b664098ad16c43dd7454e47 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:17.336079 32182 tablet.cc:1722] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:17.336423 32182 tablet.cc:1620] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:59:17.337534 32186 tablet.cc:1722] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:17.337877 32186 tablet.cc:1620] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
I20240502 07:59:17.344159 32185 tablet.cc:1722] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:17.344489 32185 tablet.cc:1620] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c: MemRowSet was empty: no flush needed.
I20240502 07:59:17.352250 22228 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=2104010124fd4c5cba27783dd01ff099] alter complete (version 2)
I20240502 07:59:17.408190 25062 tablet_service.cc:1429] Tablet server has 9 leaders and 0 scanners
I20240502 07:59:17.409744 24930 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:17.410817 24800 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:17.414564 25193 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:17.422560 24820 raft_consensus.cc:1203] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.3->[1.4-1.4] Dedup: 1.4->[]
I20240502 07:59:17.429518 22352 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:17.433990 22614 tablet_service.cc:1429] Tablet server has 22 leaders and 0 scanners
I20240502 07:59:17.445704 22225 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:17.452493 22614 tablet_service.cc:1467] Processing CreateTablet for tablet 10ae14d069e048ccba7fdfc79951cec0 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=d68deced997545d58733ac4da2118461]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:17.452571 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 10ae14d069e048ccba7fdfc79951cec0 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=d68deced997545d58733ac4da2118461]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:17.452829 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10ae14d069e048ccba7fdfc79951cec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:17.452881 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10ae14d069e048ccba7fdfc79951cec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:17.455066 32271 tablet_bootstrap.cc:492] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:17.455303 24800 tablet_service.cc:1467] Processing CreateTablet for tablet 10ae14d069e048ccba7fdfc79951cec0 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=d68deced997545d58733ac4da2118461]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:17.455566 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10ae14d069e048ccba7fdfc79951cec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:17.455561 32272 tablet_bootstrap.cc:492] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:17.455578 25062 tablet_service.cc:1467] Processing CreateTablet for tablet 52b9344c79924380a09d9ea382dbb511 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=d68deced997545d58733ac4da2118461]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:17.455830 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52b9344c79924380a09d9ea382dbb511. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:17.455868 22225 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:17.456938 32272 tablet_bootstrap.cc:654] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:17.457578 32271 tablet_bootstrap.cc:654] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:17.458500 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 26b5df5842324628b2e9bb6dc1fc36f2 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=d68deced997545d58733ac4da2118461]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:17.458731 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b5df5842324628b2e9bb6dc1fc36f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:17.460048 32274 tablet_bootstrap.cc:492] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:17.460970 32274 tablet_bootstrap.cc:654] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:17.461542 32276 tablet_bootstrap.cc:492] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:17.462009 32274 tablet_bootstrap.cc:492] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:17.462086 32274 ts_tablet_manager.cc:1367] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:17.462544 32274 raft_consensus.cc:348] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.462661 32274 raft_consensus.cc:374] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:17.462697 32274 raft_consensus.cc:729] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:17.462821 32274 consensus_queue.cc:260] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.462944 32276 tablet_bootstrap.cc:654] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:17.463135 32274 ts_tablet_manager.cc:1398] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:17.463321 32274 tablet_bootstrap.cc:492] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:17.464164 32271 tablet_bootstrap.cc:492] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:17.464259 32271 ts_tablet_manager.cc:1367] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 07:59:17.464309 32274 tablet_bootstrap.cc:654] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:17.464799 32271 raft_consensus.cc:348] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:17.464905 32271 raft_consensus.cc:374] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:17.464937 32271 raft_consensus.cc:729] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:17.465051 32271 consensus_queue.cc:260] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:17.465309 32271 ts_tablet_manager.cc:1398] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:17.465818 22481 tablet_service.cc:1467] Processing CreateTablet for tablet 52b9344c79924380a09d9ea382dbb511 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=d68deced997545d58733ac4da2118461]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:17.466526 22481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52b9344c79924380a09d9ea382dbb511. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:17.467382 32272 tablet_bootstrap.cc:492] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:17.467507 32272 ts_tablet_manager.cc:1367] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 07:59:17.468047 32272 raft_consensus.cc:348] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:17.468150 32272 raft_consensus.cc:374] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:17.468183 32272 raft_consensus.cc:729] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:17.468346 32272 consensus_queue.cc:260] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:17.468910 32274 tablet_bootstrap.cc:492] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:17.469091 32274 ts_tablet_manager.cc:1367] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:17.470136 32272 ts_tablet_manager.cc:1398] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:17.469910 32274 raft_consensus.cc:348] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:17.470225 32274 raft_consensus.cc:374] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:17.470321 32274 raft_consensus.cc:729] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:17.470477 32274 consensus_queue.cc:260] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:17.470850 32274 ts_tablet_manager.cc:1398] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:17.471197 25193 tablet_service.cc:1467] Processing CreateTablet for tablet 52b9344c79924380a09d9ea382dbb511 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=d68deced997545d58733ac4da2118461]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:17.471474 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52b9344c79924380a09d9ea382dbb511. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:17.471472 22614 tablet_service.cc:1467] Processing CreateTablet for tablet 26b5df5842324628b2e9bb6dc1fc36f2 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=d68deced997545d58733ac4da2118461]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:17.471750 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b5df5842324628b2e9bb6dc1fc36f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:17.473516 32280 tablet_bootstrap.cc:492] T 52b9344c79924380a09d9ea382dbb511 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:17.473557 25194 tablet_service.cc:1467] Processing CreateTablet for tablet 26b5df5842324628b2e9bb6dc1fc36f2 (DEFAULT_TABLE table=rebalancer_extra_table_52 [id=d68deced997545d58733ac4da2118461]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:17.473762 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b5df5842324628b2e9bb6dc1fc36f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:17.474912 32271 tablet_bootstrap.cc:492] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:17.474550 32280 tablet_bootstrap.cc:654] T 52b9344c79924380a09d9ea382dbb511 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:17.475965 32271 tablet_bootstrap.cc:654] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:17.476586 32182 tablet.cc:1722] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:17.477082 32186 tablet.cc:1722] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:17.477628 32276 tablet_bootstrap.cc:492] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:17.477783 32276 ts_tablet_manager.cc:1367] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 07:59:17.478341 32276 raft_consensus.cc:348] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:17.478545 32276 raft_consensus.cc:374] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:17.478627 32276 raft_consensus.cc:729] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:17.478807 32276 consensus_queue.cc:260] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:17.479238 32276 ts_tablet_manager.cc:1398] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:17.483629 22225 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=2104010124fd4c5cba27783dd01ff099] alter complete (version 3)
I20240502 07:59:17.487725 32185 tablet.cc:1722] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:17.506628 32281 tablet_bootstrap.cc:492] T 26b5df5842324628b2e9bb6dc1fc36f2 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:17.506945 22481 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:17.507408 22743 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:17.511220 32271 tablet_bootstrap.cc:492] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:17.511337 32271 ts_tablet_manager.cc:1367] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.036s user 0.000s sys 0.002s
I20240502 07:59:17.511948 32271 raft_consensus.cc:348] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:17.512076 32271 raft_consensus.cc:374] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:17.512122 32271 raft_consensus.cc:729] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:17.512267 32271 consensus_queue.cc:260] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:17.512655 32271 ts_tablet_manager.cc:1398] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 07:59:17.512796 32281 tablet_bootstrap.cc:654] T 26b5df5842324628b2e9bb6dc1fc36f2 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:17.519819 32280 tablet_bootstrap.cc:492] T 52b9344c79924380a09d9ea382dbb511 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:17.519913 32280 ts_tablet_manager.cc:1367] T 52b9344c79924380a09d9ea382dbb511 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.046s user 0.002s sys 0.000s
I20240502 07:59:17.520378 32280 raft_consensus.cc:348] T 52b9344c79924380a09d9ea382dbb511 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.520490 32280 raft_consensus.cc:374] T 52b9344c79924380a09d9ea382dbb511 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:17.520519 32280 raft_consensus.cc:729] T 52b9344c79924380a09d9ea382dbb511 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:17.520650 32280 consensus_queue.cc:260] T 52b9344c79924380a09d9ea382dbb511 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.520906 32280 ts_tablet_manager.cc:1398] T 52b9344c79924380a09d9ea382dbb511 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:17.540725 32281 tablet_bootstrap.cc:492] T 26b5df5842324628b2e9bb6dc1fc36f2 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:17.540841 32281 ts_tablet_manager.cc:1367] T 26b5df5842324628b2e9bb6dc1fc36f2 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.000s
I20240502 07:59:17.541395 32281 raft_consensus.cc:348] T 26b5df5842324628b2e9bb6dc1fc36f2 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:17.541523 32281 raft_consensus.cc:374] T 26b5df5842324628b2e9bb6dc1fc36f2 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:17.541561 32281 raft_consensus.cc:729] T 26b5df5842324628b2e9bb6dc1fc36f2 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:17.541675 32281 consensus_queue.cc:260] T 26b5df5842324628b2e9bb6dc1fc36f2 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:17.541975 32281 ts_tablet_manager.cc:1398] T 26b5df5842324628b2e9bb6dc1fc36f2 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:17.542431 32281 tablet_bootstrap.cc:492] T 52b9344c79924380a09d9ea382dbb511 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:17.543452 32281 tablet_bootstrap.cc:654] T 52b9344c79924380a09d9ea382dbb511 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:17.545001 32281 tablet_bootstrap.cc:492] T 52b9344c79924380a09d9ea382dbb511 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:17.545220 32281 ts_tablet_manager.cc:1367] T 52b9344c79924380a09d9ea382dbb511 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20240502 07:59:17.545709 32281 raft_consensus.cc:348] T 52b9344c79924380a09d9ea382dbb511 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.545866 32281 raft_consensus.cc:374] T 52b9344c79924380a09d9ea382dbb511 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:17.545930 32281 raft_consensus.cc:729] T 52b9344c79924380a09d9ea382dbb511 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:17.546074 32281 consensus_queue.cc:260] T 52b9344c79924380a09d9ea382dbb511 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.546427 32281 ts_tablet_manager.cc:1398] T 52b9344c79924380a09d9ea382dbb511 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:17.568495 31918 raft_consensus.cc:482] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:17.568631 31918 raft_consensus.cc:504] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.569137 31918 leader_election.cc:288] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:17.569602 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52b9344c79924380a09d9ea382dbb511" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:17.569684 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52b9344c79924380a09d9ea382dbb511" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:17.569725 22501 raft_consensus.cc:2454] T 52b9344c79924380a09d9ea382dbb511 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:17.569803 25215 raft_consensus.cc:2454] T 52b9344c79924380a09d9ea382dbb511 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:17.570252 25020 leader_election.cc:302] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:17.570438 31918 raft_consensus.cc:2781] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:17.570499 31918 raft_consensus.cc:482] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:17.570540 31918 raft_consensus.cc:3037] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:17.571668 31918 raft_consensus.cc:504] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.572031 31918 leader_election.cc:288] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:17.572314 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52b9344c79924380a09d9ea382dbb511" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:17.572369 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52b9344c79924380a09d9ea382dbb511" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:17.572430 25215 raft_consensus.cc:3037] T 52b9344c79924380a09d9ea382dbb511 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:17.572486 22501 raft_consensus.cc:3037] T 52b9344c79924380a09d9ea382dbb511 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:17.573421 22501 raft_consensus.cc:2454] T 52b9344c79924380a09d9ea382dbb511 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:17.573484 25215 raft_consensus.cc:2454] T 52b9344c79924380a09d9ea382dbb511 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:17.573741 25017 leader_election.cc:302] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:17.574036 31918 raft_consensus.cc:2781] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:17.574234 31918 raft_consensus.cc:686] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:17.574406 31918 consensus_queue.cc:237] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.576627 22221 catalog_manager.cc:5484] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:17.588169 22221 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:17.588336 22221 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:17.590827 22221 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet c89b62ba3dde4b9c9aeebbd031572030
I20240502 07:59:17.591531 22743 tablet_service.cc:1514] Processing DeleteTablet for tablet c89b62ba3dde4b9c9aeebbd031572030 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:17 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:17.591586 22481 tablet_service.cc:1514] Processing DeleteTablet for tablet c89b62ba3dde4b9c9aeebbd031572030 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:17 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:17.591706 22614 tablet_service.cc:1514] Processing DeleteTablet for tablet c89b62ba3dde4b9c9aeebbd031572030 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:17 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:17.591912 32206 tablet_replica.cc:330] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:17.592078 32206 raft_consensus.cc:2229] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:17.592219 32206 raft_consensus.cc:2258] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:17.592758 32206 ts_tablet_manager.cc:1875] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:17.595517 32206 ts_tablet_manager.cc:1888] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:17.596683 32310 tablet_replica.cc:330] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:17.596846 32310 raft_consensus.cc:2229] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:17.597059 32310 raft_consensus.cc:2258] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:17.597347 32311 tablet_replica.cc:330] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:17.597496 32311 raft_consensus.cc:2229] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:17.597612 32311 raft_consensus.cc:2258] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:17.595651 32206 log.cc:1196] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/c89b62ba3dde4b9c9aeebbd031572030
I20240502 07:59:17.598078 32310 ts_tablet_manager.cc:1875] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:17.599838 32206 ts_tablet_manager.cc:1909] T c89b62ba3dde4b9c9aeebbd031572030 P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:59:17.600565 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet c89b62ba3dde4b9c9aeebbd031572030 (table rebalancer_dynamic_table_new_name [id=2104010124fd4c5cba27783dd01ff099]) successfully deleted
I20240502 07:59:17.600816 32310 ts_tablet_manager.cc:1888] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:17.600883 32310 log.cc:1196] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/c89b62ba3dde4b9c9aeebbd031572030
I20240502 07:59:17.601913 32311 ts_tablet_manager.cc:1875] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:17.603550 32310 ts_tablet_manager.cc:1909] T c89b62ba3dde4b9c9aeebbd031572030 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:59:17.605216 32311 ts_tablet_manager.cc:1888] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:17.605379 32311 log.cc:1196] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/c89b62ba3dde4b9c9aeebbd031572030
I20240502 07:59:17.606532 32311 ts_tablet_manager.cc:1909] T c89b62ba3dde4b9c9aeebbd031572030 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:59:17.607190 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet c89b62ba3dde4b9c9aeebbd031572030 (table rebalancer_dynamic_table_new_name [id=2104010124fd4c5cba27783dd01ff099]) successfully deleted
I20240502 07:59:17.607996 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet c89b62ba3dde4b9c9aeebbd031572030 (table rebalancer_dynamic_table_new_name [id=2104010124fd4c5cba27783dd01ff099]) successfully deleted
I20240502 07:59:17.644655 31535 raft_consensus.cc:482] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:17.644754 31535 raft_consensus.cc:504] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:17.645162 31535 leader_election.cc:288] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:17.645553 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ae14d069e048ccba7fdfc79951cec0" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:17.645565 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ae14d069e048ccba7fdfc79951cec0" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:17.645689 22632 raft_consensus.cc:2454] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:17.645705 22370 raft_consensus.cc:2454] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 0.
I20240502 07:59:17.645970 24757 leader_election.cc:302] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:17.646226 31535 raft_consensus.cc:2781] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:17.646281 31535 raft_consensus.cc:482] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:17.646315 31535 raft_consensus.cc:3037] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:17.647334 31535 raft_consensus.cc:504] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:17.647650 31535 leader_election.cc:288] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:17.648149 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ae14d069e048ccba7fdfc79951cec0" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:17.648157 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ae14d069e048ccba7fdfc79951cec0" candidate_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:17.648264 22370 raft_consensus.cc:3037] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:17.648267 22632 raft_consensus.cc:3037] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:17.649121 22370 raft_consensus.cc:2454] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:17.649364 22632 raft_consensus.cc:2454] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b4e7eeb7f3f42119c0363a79febc54b in term 1.
I20240502 07:59:17.649410 24757 leader_election.cc:302] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:17.649561 31535 raft_consensus.cc:2781] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:17.649643 31535 raft_consensus.cc:686] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Becoming Leader. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Running, Role: LEADER
I20240502 07:59:17.649757 31535 consensus_queue.cc:237] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:17.652132 22230 catalog_manager.cc:5484] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133). New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:17.693676 22226 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:17.701459 22352 tablet_service.cc:1467] Processing CreateTablet for tablet da8590dc66034ecebbbf85c90dacc233 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=05c13c3ecc90432a8463a422abf0a71a]), partition=
I20240502 07:59:17.701619 22481 tablet_service.cc:1467] Processing CreateTablet for tablet da8590dc66034ecebbbf85c90dacc233 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=05c13c3ecc90432a8463a422abf0a71a]), partition=
I20240502 07:59:17.701752 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da8590dc66034ecebbbf85c90dacc233. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:17.701843 22481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da8590dc66034ecebbbf85c90dacc233. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:17.703521 32280 tablet_bootstrap.cc:492] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:17.703688 25063 tablet_service.cc:1467] Processing CreateTablet for tablet da8590dc66034ecebbbf85c90dacc233 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=05c13c3ecc90432a8463a422abf0a71a]), partition=
I20240502 07:59:17.703907 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da8590dc66034ecebbbf85c90dacc233. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:17.704490 32280 tablet_bootstrap.cc:654] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:17.705085 32272 tablet_bootstrap.cc:492] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:17.705689 32280 tablet_bootstrap.cc:492] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:17.705775 32280 ts_tablet_manager.cc:1367] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:17.705894 32274 tablet_bootstrap.cc:492] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:17.706097 32272 tablet_bootstrap.cc:654] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:17.706243 32280 raft_consensus.cc:348] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.706359 32280 raft_consensus.cc:374] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:17.706388 32280 raft_consensus.cc:729] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:17.706502 32280 consensus_queue.cc:260] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.706773 32274 tablet_bootstrap.cc:654] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:17.706815 32280 ts_tablet_manager.cc:1398] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:17.707790 32274 tablet_bootstrap.cc:492] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:17.707880 32274 ts_tablet_manager.cc:1367] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:17.708320 32274 raft_consensus.cc:348] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.708426 32274 raft_consensus.cc:374] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:17.708460 32274 raft_consensus.cc:729] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:17.708565 32274 consensus_queue.cc:260] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.708812 32274 ts_tablet_manager.cc:1398] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:17.714677 32272 tablet_bootstrap.cc:492] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:17.714793 32272 ts_tablet_manager.cc:1367] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 07:59:17.715271 32272 raft_consensus.cc:348] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.715359 32272 raft_consensus.cc:374] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:17.715379 32272 raft_consensus.cc:729] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:17.715513 32272 consensus_queue.cc:260] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.715766 32272 ts_tablet_manager.cc:1398] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:17.730376 32292 raft_consensus.cc:482] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:17.730506 32292 raft_consensus.cc:504] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:17.730913 32292 leader_election.cc:288] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:17.731230 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b5df5842324628b2e9bb6dc1fc36f2" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:17.731372 25215 raft_consensus.cc:2454] T 26b5df5842324628b2e9bb6dc1fc36f2 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:17.731606 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b5df5842324628b2e9bb6dc1fc36f2" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:17.731698 25020 leader_election.cc:302] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:17.731736 22634 raft_consensus.cc:2454] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:17.731913 32292 raft_consensus.cc:2781] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:17.731978 32292 raft_consensus.cc:482] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:17.732030 32292 raft_consensus.cc:3037] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:17.732944 32292 raft_consensus.cc:504] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:17.733247 32292 leader_election.cc:288] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:17.733801 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b5df5842324628b2e9bb6dc1fc36f2" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:17.733922 25215 raft_consensus.cc:3037] T 26b5df5842324628b2e9bb6dc1fc36f2 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:17.734889 25215 raft_consensus.cc:2454] T 26b5df5842324628b2e9bb6dc1fc36f2 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:17.735272 25020 leader_election.cc:302] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:17.735435 32292 raft_consensus.cc:2781] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:17.735510 32292 raft_consensus.cc:686] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:17.735633 32292 consensus_queue.cc:237] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:17.736276 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b5df5842324628b2e9bb6dc1fc36f2" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:17.736387 22634 raft_consensus.cc:3037] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:17.737143 22634 raft_consensus.cc:2454] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:17.740021 22228 catalog_manager.cc:5484] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:17.770880 31918 raft_consensus.cc:482] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:17.771008 31918 raft_consensus.cc:504] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.771418 31918 leader_election.cc:288] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:17.771958 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da8590dc66034ecebbbf85c90dacc233" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:17.772086 22370 raft_consensus.cc:2454] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:17.772352 25020 leader_election.cc:302] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:17.772382 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da8590dc66034ecebbbf85c90dacc233" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:17.772497 31918 raft_consensus.cc:2781] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:17.772505 22501 raft_consensus.cc:2454] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:17.772540 31918 raft_consensus.cc:482] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:17.772575 31918 raft_consensus.cc:3037] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:17.773767 31918 raft_consensus.cc:504] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.774122 31918 leader_election.cc:288] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:17.774698 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da8590dc66034ecebbbf85c90dacc233" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:17.774824 22501 raft_consensus.cc:3037] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:17.774852 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da8590dc66034ecebbbf85c90dacc233" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:17.774957 22370 raft_consensus.cc:3037] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:17.775782 22501 raft_consensus.cc:2454] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:17.775873 22370 raft_consensus.cc:2454] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:17.776062 25017 leader_election.cc:302] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:17.776609 31918 raft_consensus.cc:2781] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:17.776692 31918 raft_consensus.cc:686] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:17.776818 31918 consensus_queue.cc:237] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:17.777753 24930 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:17.780637 22481 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:17.782737 25063 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:17.784556 22352 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:17.786659 22227 catalog_manager.cc:5484] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:17.795941 25193 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:17.797727 22743 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:17.801070 22614 tablet_service.cc:1429] Tablet server has 22 leaders and 0 scanners
I20240502 07:59:17.802184 24800 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:17.853678 22634 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d61cb2b6dce1409bb30d1c68c970dc22"
dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:35384
I20240502 07:59:17.853819 22634 raft_consensus.cc:595] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:17.899915 31410 raft_consensus.cc:979] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f: : Instructing follower fabddb16c85944648a710ec52fc7244c to start an election
I20240502 07:59:17.900033 31410 raft_consensus.cc:1067] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Signalling peer fabddb16c85944648a710ec52fc7244c to start an election
I20240502 07:59:17.900493 22501 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d61cb2b6dce1409bb30d1c68c970dc22"
dest_uuid: "fabddb16c85944648a710ec52fc7244c"
from {username='slave'} at 127.15.129.131:46727
I20240502 07:59:17.900656 22501 raft_consensus.cc:482] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:17.900732 22501 raft_consensus.cc:3037] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:17.901700 22501 raft_consensus.cc:504] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:17.902334 22501 leader_election.cc:288] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 2 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:17.903491 24819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61cb2b6dce1409bb30d1c68c970dc22" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:17.903618 24819 raft_consensus.cc:3037] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:17.904752 24819 raft_consensus.cc:2454] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 2.
I20240502 07:59:17.905714 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61cb2b6dce1409bb30d1c68c970dc22" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:17.905839 22764 raft_consensus.cc:3037] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:17.906896 22764 raft_consensus.cc:2454] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 2.
I20240502 07:59:17.907514 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61cb2b6dce1409bb30d1c68c970dc22" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:17.907639 22634 raft_consensus.cc:3032] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:17.907681 22634 raft_consensus.cc:729] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:59:17.907840 22634 consensus_queue.cc:260] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:17.908092 22634 raft_consensus.cc:3037] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:17.908685 22439 leader_election.cc:302] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:17.908864 31420 raft_consensus.cc:2781] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:17.908948 31420 raft_consensus.cc:686] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [term 2 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:59:17.909050 22634 raft_consensus.cc:2454] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 2.
I20240502 07:59:17.909077 31420 consensus_queue.cc:237] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:17.912820 22229 catalog_manager.cc:5484] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 1 to 2, leader changed from e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 2 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:17.935308 22224 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:17.943171 22501 raft_consensus.cc:1261] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:17.943573 32292 consensus_queue.cc:1035] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:17.944077 22370 raft_consensus.cc:1261] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:17.944558 32292 consensus_queue.cc:1035] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:17.946692 32185 tablet.cc:1722] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:17.946877 22230 catalog_manager.cc:5576] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625) does not have the latest schema for tablet da8590dc66034ecebbbf85c90dacc233 (table rebalancer_dynamic_table [id=05c13c3ecc90432a8463a422abf0a71a]). Expected version 1 got 0
I20240502 07:59:17.947026 32185 tablet.cc:1620] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c: MemRowSet was empty: no flush needed.
I20240502 07:59:17.949914 32441 tablet.cc:1722] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:17.950709 32441 tablet.cc:1620] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:17.954627 22226 catalog_manager.cc:5576] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535) does not have the latest schema for tablet da8590dc66034ecebbbf85c90dacc233 (table rebalancer_dynamic_table [id=05c13c3ecc90432a8463a422abf0a71a]). Expected version 1 got 0
I20240502 07:59:17.958665 22229 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=05c13c3ecc90432a8463a422abf0a71a] alter complete (version 1)
I20240502 07:59:17.963080 32438 tablet.cc:1722] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:17.963429 32438 tablet.cc:1620] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:17.985841 24930 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:17.991441 24800 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:17.993125 25193 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:17.993734 25063 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:18.001497 22743 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:18.004184 22614 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 07:59:18.019928 22352 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:18.026321 32292 consensus_queue.cc:1035] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:18.030412 22481 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:18.044574 31918 consensus_queue.cc:1035] T 52b9344c79924380a09d9ea382dbb511 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:18.062961 22222 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:18.069182 32438 tablet.cc:1722] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:18.069561 32438 tablet.cc:1620] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:18.070730 32185 tablet.cc:1722] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:18.070847 32441 tablet.cc:1722] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:18.071060 32185 tablet.cc:1620] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c: MemRowSet was empty: no flush needed.
I20240502 07:59:18.071146 32441 tablet.cc:1620] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:18.076409 22229 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=05c13c3ecc90432a8463a422abf0a71a] alter complete (version 2)
I20240502 07:59:18.124935 31918 consensus_queue.cc:1035] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:18.131366 32292 consensus_queue.cc:1035] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:18.153878 32178 consensus_queue.cc:1035] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:18.156802 32178 consensus_queue.cc:1035] T 10ae14d069e048ccba7fdfc79951cec0 P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:18.178889 22230 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:18.183748 32438 tablet.cc:1722] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:18.184386 32185 tablet.cc:1722] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:18.187366 32441 tablet.cc:1722] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:18.189189 22230 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=05c13c3ecc90432a8463a422abf0a71a] alter complete (version 3)
I20240502 07:59:18.254012 22222 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_54"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:18.260123 24930 tablet_service.cc:1467] Processing CreateTablet for tablet 07c25335b5e44744a67404fe1cd227b2 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=8bbeaa9f10a347fe87e227b0112cc204]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:18.260210 22614 tablet_service.cc:1467] Processing CreateTablet for tablet 07c25335b5e44744a67404fe1cd227b2 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=8bbeaa9f10a347fe87e227b0112cc204]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:18.260458 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07c25335b5e44744a67404fe1cd227b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:18.260506 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07c25335b5e44744a67404fe1cd227b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:18.260516 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 07c25335b5e44744a67404fe1cd227b2 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=8bbeaa9f10a347fe87e227b0112cc204]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:18.260769 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07c25335b5e44744a67404fe1cd227b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:18.260936 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 31fb10d4041c43b2b8a0f78866a53278 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=8bbeaa9f10a347fe87e227b0112cc204]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:18.261183 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31fb10d4041c43b2b8a0f78866a53278. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:18.263398 32466 tablet_bootstrap.cc:492] T 31fb10d4041c43b2b8a0f78866a53278 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:18.264428 32465 tablet_bootstrap.cc:492] T 07c25335b5e44744a67404fe1cd227b2 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:18.264523 25193 tablet_service.cc:1467] Processing CreateTablet for tablet 31fb10d4041c43b2b8a0f78866a53278 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=8bbeaa9f10a347fe87e227b0112cc204]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:18.264755 32466 tablet_bootstrap.cc:654] T 31fb10d4041c43b2b8a0f78866a53278 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:18.264761 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31fb10d4041c43b2b8a0f78866a53278. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:18.266052 32465 tablet_bootstrap.cc:654] T 07c25335b5e44744a67404fe1cd227b2 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:18.266270 32466 tablet_bootstrap.cc:492] T 31fb10d4041c43b2b8a0f78866a53278 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:18.266364 32466 ts_tablet_manager.cc:1367] T 31fb10d4041c43b2b8a0f78866a53278 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:18.266834 32464 tablet_bootstrap.cc:492] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:18.266821 32466 raft_consensus.cc:348] T 31fb10d4041c43b2b8a0f78866a53278 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:18.266945 32466 raft_consensus.cc:374] T 31fb10d4041c43b2b8a0f78866a53278 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:18.266989 32466 raft_consensus.cc:729] T 31fb10d4041c43b2b8a0f78866a53278 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:18.267083 32469 tablet_bootstrap.cc:492] T 31fb10d4041c43b2b8a0f78866a53278 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:18.267105 32466 consensus_queue.cc:260] T 31fb10d4041c43b2b8a0f78866a53278 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:18.267210 22481 tablet_service.cc:1467] Processing CreateTablet for tablet 31fb10d4041c43b2b8a0f78866a53278 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=8bbeaa9f10a347fe87e227b0112cc204]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:18.267417 32466 ts_tablet_manager.cc:1398] T 31fb10d4041c43b2b8a0f78866a53278 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:18.267478 22481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31fb10d4041c43b2b8a0f78866a53278. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:18.268600 32465 tablet_bootstrap.cc:492] T 07c25335b5e44744a67404fe1cd227b2 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:18.268751 32465 ts_tablet_manager.cc:1367] T 07c25335b5e44744a67404fe1cd227b2 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:18.269017 32469 tablet_bootstrap.cc:654] T 31fb10d4041c43b2b8a0f78866a53278 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:18.269373 32465 raft_consensus.cc:348] T 07c25335b5e44744a67404fe1cd227b2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:18.269516 32465 raft_consensus.cc:374] T 07c25335b5e44744a67404fe1cd227b2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:18.269587 32465 raft_consensus.cc:729] T 07c25335b5e44744a67404fe1cd227b2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:18.270171 32471 tablet_bootstrap.cc:492] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:18.271255 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 837daa4ba2f1429b8cc5ae163707a343 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=8bbeaa9f10a347fe87e227b0112cc204]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:18.271381 32471 tablet_bootstrap.cc:654] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:18.271585 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 837daa4ba2f1429b8cc5ae163707a343. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:18.272136 32465 consensus_queue.cc:260] T 07c25335b5e44744a67404fe1cd227b2 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:18.272686 32465 ts_tablet_manager.cc:1398] T 07c25335b5e44744a67404fe1cd227b2 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:18.273802 32464 tablet_bootstrap.cc:654] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:18.274974 22614 tablet_service.cc:1467] Processing CreateTablet for tablet 837daa4ba2f1429b8cc5ae163707a343 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=8bbeaa9f10a347fe87e227b0112cc204]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:18.275346 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 837daa4ba2f1429b8cc5ae163707a343. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:18.275580 24930 tablet_service.cc:1467] Processing CreateTablet for tablet 837daa4ba2f1429b8cc5ae163707a343 (DEFAULT_TABLE table=rebalancer_extra_table_54 [id=8bbeaa9f10a347fe87e227b0112cc204]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:18.275822 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 837daa4ba2f1429b8cc5ae163707a343. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:18.276449 32464 tablet_bootstrap.cc:492] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:18.276576 32464 ts_tablet_manager.cc:1367] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 07:59:18.277092 32464 raft_consensus.cc:348] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:18.277208 32464 raft_consensus.cc:374] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:18.277240 32464 raft_consensus.cc:729] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:18.277419 32464 consensus_queue.cc:260] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:18.277899 32463 tablet_bootstrap.cc:492] T 07c25335b5e44744a67404fe1cd227b2 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:18.278755 32464 ts_tablet_manager.cc:1398] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:18.279062 32463 tablet_bootstrap.cc:654] T 07c25335b5e44744a67404fe1cd227b2 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:18.280303 32463 tablet_bootstrap.cc:492] T 07c25335b5e44744a67404fe1cd227b2 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:18.280448 32463 ts_tablet_manager.cc:1367] T 07c25335b5e44744a67404fe1cd227b2 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:18.280508 32469 tablet_bootstrap.cc:492] T 31fb10d4041c43b2b8a0f78866a53278 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:18.280627 32469 ts_tablet_manager.cc:1367] T 31fb10d4041c43b2b8a0f78866a53278 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 07:59:18.280963 32463 raft_consensus.cc:348] T 07c25335b5e44744a67404fe1cd227b2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:18.281059 32463 raft_consensus.cc:374] T 07c25335b5e44744a67404fe1cd227b2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:18.281090 32463 raft_consensus.cc:729] T 07c25335b5e44744a67404fe1cd227b2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:18.281193 32469 raft_consensus.cc:348] T 31fb10d4041c43b2b8a0f78866a53278 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:18.281311 32469 raft_consensus.cc:374] T 31fb10d4041c43b2b8a0f78866a53278 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:18.281390 32469 raft_consensus.cc:729] T 31fb10d4041c43b2b8a0f78866a53278 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:18.281519 32469 consensus_queue.cc:260] T 31fb10d4041c43b2b8a0f78866a53278 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:18.281730 32464 tablet_bootstrap.cc:492] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:18.281839 32469 ts_tablet_manager.cc:1398] T 31fb10d4041c43b2b8a0f78866a53278 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:18.282212 32463 consensus_queue.cc:260] T 07c25335b5e44744a67404fe1cd227b2 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:18.282565 32463 ts_tablet_manager.cc:1398] T 07c25335b5e44744a67404fe1cd227b2 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:18.282716 32463 tablet_bootstrap.cc:492] T 837daa4ba2f1429b8cc5ae163707a343 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:18.282938 32464 tablet_bootstrap.cc:654] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:18.283869 32463 tablet_bootstrap.cc:654] T 837daa4ba2f1429b8cc5ae163707a343 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:18.284328 32471 tablet_bootstrap.cc:492] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:18.284459 32471 ts_tablet_manager.cc:1367] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 07:59:18.285032 32471 raft_consensus.cc:348] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:18.285174 32471 raft_consensus.cc:374] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:18.285219 32471 raft_consensus.cc:729] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:18.285382 32471 consensus_queue.cc:260] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:18.285641 32473 tablet_bootstrap.cc:492] T 837daa4ba2f1429b8cc5ae163707a343 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:18.285710 32471 ts_tablet_manager.cc:1398] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:18.287747 32473 tablet_bootstrap.cc:654] T 837daa4ba2f1429b8cc5ae163707a343 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:18.288877 32473 tablet_bootstrap.cc:492] T 837daa4ba2f1429b8cc5ae163707a343 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:18.288959 32473 ts_tablet_manager.cc:1367] T 837daa4ba2f1429b8cc5ae163707a343 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:18.289485 32473 raft_consensus.cc:348] T 837daa4ba2f1429b8cc5ae163707a343 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:18.289582 32473 raft_consensus.cc:374] T 837daa4ba2f1429b8cc5ae163707a343 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:18.289611 32473 raft_consensus.cc:729] T 837daa4ba2f1429b8cc5ae163707a343 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:18.289618 32464 tablet_bootstrap.cc:492] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:18.289738 32473 consensus_queue.cc:260] T 837daa4ba2f1429b8cc5ae163707a343 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:18.289886 32464 ts_tablet_manager.cc:1367] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20240502 07:59:18.290037 32473 ts_tablet_manager.cc:1398] T 837daa4ba2f1429b8cc5ae163707a343 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:18.292043 32463 tablet_bootstrap.cc:492] T 837daa4ba2f1429b8cc5ae163707a343 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:18.292148 32463 ts_tablet_manager.cc:1367] T 837daa4ba2f1429b8cc5ae163707a343 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20240502 07:59:18.292639 32463 raft_consensus.cc:348] T 837daa4ba2f1429b8cc5ae163707a343 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:18.292752 32463 raft_consensus.cc:374] T 837daa4ba2f1429b8cc5ae163707a343 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:18.292781 32463 raft_consensus.cc:729] T 837daa4ba2f1429b8cc5ae163707a343 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:18.292889 32463 consensus_queue.cc:260] T 837daa4ba2f1429b8cc5ae163707a343 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:18.293136 32463 ts_tablet_manager.cc:1398] T 837daa4ba2f1429b8cc5ae163707a343 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:18.294261 22224 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:18.294376 22224 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:18.295626 32464 raft_consensus.cc:348] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:18.295737 32464 raft_consensus.cc:374] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:18.295770 32464 raft_consensus.cc:729] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:18.295884 32464 consensus_queue.cc:260] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:18.296175 32464 ts_tablet_manager.cc:1398] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.006s user 0.000s sys 0.000s
I20240502 07:59:18.296984 32270 raft_consensus.cc:482] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:18.297075 32270 raft_consensus.cc:504] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:18.297518 32270 leader_election.cc:288] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:18.298146 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837daa4ba2f1429b8cc5ae163707a343" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:18.298421 25084 raft_consensus.cc:2454] T 837daa4ba2f1429b8cc5ae163707a343 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:18.298733 24885 leader_election.cc:302] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
I20240502 07:59:18.299093 32485 raft_consensus.cc:2781] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:18.299067 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837daa4ba2f1429b8cc5ae163707a343" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:18.299145 32485 raft_consensus.cc:482] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:18.299186 32485 raft_consensus.cc:3037] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:18.299208 22632 raft_consensus.cc:2454] T 837daa4ba2f1429b8cc5ae163707a343 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:18.299645 22224 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet da8590dc66034ecebbbf85c90dacc233
I20240502 07:59:18.300263 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet da8590dc66034ecebbbf85c90dacc233 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:18 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:18.300411 32485 raft_consensus.cc:504] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:18.302672 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837daa4ba2f1429b8cc5ae163707a343" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:18.302794 22632 raft_consensus.cc:3037] T 837daa4ba2f1429b8cc5ae163707a343 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:18.303850 22632 raft_consensus.cc:2454] T 837daa4ba2f1429b8cc5ae163707a343 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:18.304359 24888 leader_election.cc:302] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:18.304598 32485 leader_election.cc:288] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:18.304622 32477 raft_consensus.cc:2781] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:18.304708 32477 raft_consensus.cc:686] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:18.304854 32477 consensus_queue.cc:237] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:18.305374 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837daa4ba2f1429b8cc5ae163707a343" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:18.305500 25084 raft_consensus.cc:3037] T 837daa4ba2f1429b8cc5ae163707a343 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:18.306448 25084 raft_consensus.cc:2454] T 837daa4ba2f1429b8cc5ae163707a343 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:18.309002 22352 tablet_service.cc:1514] Processing DeleteTablet for tablet da8590dc66034ecebbbf85c90dacc233 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:18 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:18.309093 22481 tablet_service.cc:1514] Processing DeleteTablet for tablet da8590dc66034ecebbbf85c90dacc233 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:18 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:18.309403 32494 tablet_replica.cc:330] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:18.309558 32494 raft_consensus.cc:2229] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:18.310081 32486 tablet_replica.cc:330] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:18.310222 32486 raft_consensus.cc:2229] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:18.310515 32486 raft_consensus.cc:2258] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:18.310621 32494 raft_consensus.cc:2258] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:18.311679 32486 ts_tablet_manager.cc:1875] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:18.311772 32494 ts_tablet_manager.cc:1875] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:18.312362 22224 catalog_manager.cc:5484] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:18.314261 32494 ts_tablet_manager.cc:1888] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:18.314456 32495 tablet_replica.cc:330] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:18.314586 32495 raft_consensus.cc:2229] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:18.314697 32495 raft_consensus.cc:2258] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:18.314839 32494 log.cc:1196] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/da8590dc66034ecebbbf85c90dacc233
I20240502 07:59:18.315227 32494 ts_tablet_manager.cc:1909] T da8590dc66034ecebbbf85c90dacc233 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:59:18.315989 32495 ts_tablet_manager.cc:1875] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:18.316588 32486 ts_tablet_manager.cc:1888] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:18.316663 32486 log.cc:1196] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/da8590dc66034ecebbbf85c90dacc233
I20240502 07:59:18.316913 32486 ts_tablet_manager.cc:1909] T da8590dc66034ecebbbf85c90dacc233 P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:59:18.317728 32495 ts_tablet_manager.cc:1888] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:18.317791 32495 log.cc:1196] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/da8590dc66034ecebbbf85c90dacc233
I20240502 07:59:18.317878 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet da8590dc66034ecebbbf85c90dacc233 (table rebalancer_dynamic_table_new_name [id=05c13c3ecc90432a8463a422abf0a71a]) successfully deleted
I20240502 07:59:18.318044 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet da8590dc66034ecebbbf85c90dacc233 (table rebalancer_dynamic_table_new_name [id=05c13c3ecc90432a8463a422abf0a71a]) successfully deleted
I20240502 07:59:18.318064 32495 ts_tablet_manager.cc:1909] T da8590dc66034ecebbbf85c90dacc233 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:59:18.318634 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet da8590dc66034ecebbbf85c90dacc233 (table rebalancer_dynamic_table_new_name [id=05c13c3ecc90432a8463a422abf0a71a]) successfully deleted
I20240502 07:59:18.336537 32485 raft_consensus.cc:482] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:18.336660 32485 raft_consensus.cc:504] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:18.337124 32485 leader_election.cc:288] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:18.337543 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07c25335b5e44744a67404fe1cd227b2" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:18.337678 22632 raft_consensus.cc:2454] T 07c25335b5e44744a67404fe1cd227b2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:18.337957 24888 leader_election.cc:302] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:18.338115 32485 raft_consensus.cc:2781] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:18.338163 32485 raft_consensus.cc:482] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:18.338204 32485 raft_consensus.cc:3037] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:18.338335 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07c25335b5e44744a67404fe1cd227b2" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:18.338471 22370 raft_consensus.cc:2454] T 07c25335b5e44744a67404fe1cd227b2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:18.340215 32485 raft_consensus.cc:504] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:18.340610 32485 leader_election.cc:288] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:18.340879 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07c25335b5e44744a67404fe1cd227b2" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:18.340924 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07c25335b5e44744a67404fe1cd227b2" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:18.341001 22370 raft_consensus.cc:3037] T 07c25335b5e44744a67404fe1cd227b2 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:18.341022 22632 raft_consensus.cc:3037] T 07c25335b5e44744a67404fe1cd227b2 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:18.341858 22370 raft_consensus.cc:2454] T 07c25335b5e44744a67404fe1cd227b2 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:18.341856 22632 raft_consensus.cc:2454] T 07c25335b5e44744a67404fe1cd227b2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:18.342118 24888 leader_election.cc:302] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:18.342289 32485 raft_consensus.cc:2781] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:18.342370 32485 raft_consensus.cc:686] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:18.342525 32485 consensus_queue.cc:237] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:18.345194 22229 catalog_manager.cc:5484] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:18.367308 24819 raft_consensus.cc:1261] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b [term 2 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:18.367870 31420 consensus_queue.cc:1035] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:18.408918 22764 raft_consensus.cc:1261] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:18.409307 31420 consensus_queue.cc:1035] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:18.410159 22224 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:18.412698 22632 raft_consensus.cc:1261] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 2 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:18.413261 31770 consensus_queue.cc:1035] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:18.418879 24930 tablet_service.cc:1467] Processing CreateTablet for tablet cc8e7771b0bd41b38b740266a7559a57 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e5095259076a4a338b35330408063c71]), partition=
I20240502 07:59:18.419140 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc8e7771b0bd41b38b740266a7559a57. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:18.419113 25193 tablet_service.cc:1467] Processing CreateTablet for tablet cc8e7771b0bd41b38b740266a7559a57 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e5095259076a4a338b35330408063c71]), partition=
I20240502 07:59:18.419368 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc8e7771b0bd41b38b740266a7559a57. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:18.421413 32469 tablet_bootstrap.cc:492] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:18.421495 32464 tablet_bootstrap.cc:492] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:18.422230 32469 tablet_bootstrap.cc:654] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:18.422331 32464 tablet_bootstrap.cc:654] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:18.423008 22352 tablet_service.cc:1467] Processing CreateTablet for tablet cc8e7771b0bd41b38b740266a7559a57 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=e5095259076a4a338b35330408063c71]), partition=
I20240502 07:59:18.423247 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc8e7771b0bd41b38b740266a7559a57. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:18.423323 32469 tablet_bootstrap.cc:492] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:18.423393 32469 ts_tablet_manager.cc:1367] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:18.423735 32469 raft_consensus.cc:348] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:18.423807 32469 raft_consensus.cc:374] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:18.423827 32469 raft_consensus.cc:729] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:18.423934 32469 consensus_queue.cc:260] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:18.424232 32469 ts_tablet_manager.cc:1398] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:18.425294 32465 tablet_bootstrap.cc:492] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:18.426211 32465 tablet_bootstrap.cc:654] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:18.429729 32465 tablet_bootstrap.cc:492] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:18.429812 32465 ts_tablet_manager.cc:1367] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:18.430227 32465 raft_consensus.cc:348] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:18.430349 32465 raft_consensus.cc:374] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:18.430377 32465 raft_consensus.cc:729] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:18.430478 32465 consensus_queue.cc:260] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:18.430724 32465 ts_tablet_manager.cc:1398] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:18.431886 32464 tablet_bootstrap.cc:492] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:18.431972 32464 ts_tablet_manager.cc:1367] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.001s
I20240502 07:59:18.432909 32464 raft_consensus.cc:348] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:18.433005 32464 raft_consensus.cc:374] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:18.433030 32464 raft_consensus.cc:729] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:18.433115 32464 consensus_queue.cc:260] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:18.434264 32464 ts_tablet_manager.cc:1398] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:18.507553 32529 raft_consensus.cc:482] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:18.507705 32529 raft_consensus.cc:504] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:18.508152 32529 leader_election.cc:288] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:18.508517 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31fb10d4041c43b2b8a0f78866a53278" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:18.508637 25213 raft_consensus.cc:2454] T 31fb10d4041c43b2b8a0f78866a53278 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:18.508718 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31fb10d4041c43b2b8a0f78866a53278" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:18.508881 22439 leader_election.cc:302] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:18.508878 22764 raft_consensus.cc:2454] T 31fb10d4041c43b2b8a0f78866a53278 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:18.509025 32529 raft_consensus.cc:2781] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:18.509083 32529 raft_consensus.cc:482] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:18.509130 32529 raft_consensus.cc:3037] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:18.510043 32529 raft_consensus.cc:504] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:18.510407 32529 leader_election.cc:288] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:18.510684 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31fb10d4041c43b2b8a0f78866a53278" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:18.510773 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31fb10d4041c43b2b8a0f78866a53278" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:18.510849 22764 raft_consensus.cc:3037] T 31fb10d4041c43b2b8a0f78866a53278 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:18.510898 25213 raft_consensus.cc:3037] T 31fb10d4041c43b2b8a0f78866a53278 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:18.511881 25213 raft_consensus.cc:2454] T 31fb10d4041c43b2b8a0f78866a53278 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:18.511881 22764 raft_consensus.cc:2454] T 31fb10d4041c43b2b8a0f78866a53278 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:18.512168 22439 leader_election.cc:302] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:18.512327 32529 raft_consensus.cc:2781] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:18.512413 32529 raft_consensus.cc:686] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:59:18.512544 32529 consensus_queue.cc:237] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:18.514792 22224 catalog_manager.cc:5484] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:18.537233 32477 raft_consensus.cc:482] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:18.537355 32477 raft_consensus.cc:504] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:18.537742 32477 leader_election.cc:288] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:18.538153 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc8e7771b0bd41b38b740266a7559a57" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:18.538286 25213 raft_consensus.cc:2454] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:18.538549 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc8e7771b0bd41b38b740266a7559a57" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:18.538686 22370 raft_consensus.cc:2454] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:18.539050 24888 leader_election.cc:302] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:18.539310 32477 raft_consensus.cc:2781] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:18.539373 32477 raft_consensus.cc:482] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:18.539418 32477 raft_consensus.cc:3037] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:18.540553 32477 raft_consensus.cc:504] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:18.540931 32477 leader_election.cc:288] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:18.541735 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc8e7771b0bd41b38b740266a7559a57" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:18.541843 25213 raft_consensus.cc:3037] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:18.542057 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc8e7771b0bd41b38b740266a7559a57" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:18.542198 22370 raft_consensus.cc:3037] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:18.542773 25213 raft_consensus.cc:2454] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:18.543208 22370 raft_consensus.cc:2454] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:18.543712 24888 leader_election.cc:302] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:18.543947 32477 raft_consensus.cc:2781] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:18.544034 32477 raft_consensus.cc:686] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:18.544162 32477 consensus_queue.cc:237] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:18.545387 25063 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:18.545711 25193 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:18.546876 22230 catalog_manager.cc:5484] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:18.560720 22614 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 07:59:18.560946 22352 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:18.570574 22481 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:18.570997 24800 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:18.580762 24930 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:18.599303 22743 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:18.654609 22501 consensus_queue.cc:237] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:18.655833 22764 raft_consensus.cc:1261] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 07:59:18.655990 24819 raft_consensus.cc:1261] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b [term 2 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 07:59:18.656229 31420 consensus_queue.cc:1035] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:18.656718 31420 consensus_queue.cc:1035] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:18.658094 31420 raft_consensus.cc:2932] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c [term 2 LEADER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:18.658380 22764 raft_consensus.cc:2932] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:18.658679 24819 raft_consensus.cc:2932] T d61cb2b6dce1409bb30d1c68c970dc22 P 4b4e7eeb7f3f42119c0363a79febc54b [term 2 FOLLOWER]: Committing config change with OpId 2.6: config changed from index 4 to 6, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:18.661638 22230 catalog_manager.cc:5484] T d61cb2b6dce1409bb30d1c68c970dc22 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index 4 to 6, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New cstate: current_term: 2 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:18.664839 22614 tablet_service.cc:1514] Processing DeleteTablet for tablet d61cb2b6dce1409bb30d1c68c970dc22 with delete_type TABLET_DATA_TOMBSTONED (TS e51b10af49484aaa8f0267aebc1bf27f not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:18.665195 32571 tablet_replica.cc:330] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:18.665479 32571 raft_consensus.cc:2229] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 2 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:18.665720 32571 raft_consensus.cc:2258] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f [term 2 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:18.666847 32571 ts_tablet_manager.cc:1875] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:18.666994 22226 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:18.669359 32571 ts_tablet_manager.cc:1888] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.5
I20240502 07:59:18.669471 32571 log.cc:1196] T d61cb2b6dce1409bb30d1c68c970dc22 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/d61cb2b6dce1409bb30d1c68c970dc22
I20240502 07:59:18.670011 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet d61cb2b6dce1409bb30d1c68c970dc22 (table rebalance_test_table_1 [id=be3f9b3a6e5147b0afe27972b50219ce]) successfully deleted
I20240502 07:59:18.671409 25215 raft_consensus.cc:1261] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:18.671502 22370 raft_consensus.cc:1261] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:18.671787 32485 consensus_queue.cc:1035] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:18.671921 32477 consensus_queue.cc:1035] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:18.675163 22226 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet cc8e7771b0bd41b38b740266a7559a57 (table rebalancer_dynamic_table [id=e5095259076a4a338b35330408063c71]). Expected version 1 got 0
I20240502 07:59:18.675451 32576 tablet.cc:1722] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:18.675930 32576 tablet.cc:1620] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:18.676066 32441 tablet.cc:1722] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:18.676374 32441 tablet.cc:1620] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:18.676621 22224 catalog_manager.cc:5576] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535) does not have the latest schema for tablet cc8e7771b0bd41b38b740266a7559a57 (table rebalancer_dynamic_table [id=e5095259076a4a338b35330408063c71]). Expected version 1 got 0
I20240502 07:59:18.680330 32577 tablet.cc:1722] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:18.680616 32577 tablet.cc:1620] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:18.681020 22224 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=e5095259076a4a338b35330408063c71] alter complete (version 1)
I20240502 07:59:18.682435 22224 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet cc8e7771b0bd41b38b740266a7559a57 (table rebalancer_dynamic_table [id=e5095259076a4a338b35330408063c71]). Expected version 1 got 0
I20240502 07:59:18.784262 22226 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:18.789716 32576 tablet.cc:1722] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:18.790102 32576 tablet.cc:1620] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:18.790300 32441 tablet.cc:1722] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:18.790589 32441 tablet.cc:1620] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:18.792773 32577 tablet.cc:1722] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:18.793357 32577 tablet.cc:1620] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:18.796895 22226 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=e5095259076a4a338b35330408063c71] alter complete (version 2)
I20240502 07:59:18.816972 32270 consensus_queue.cc:1035] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:18.819947 32270 consensus_queue.cc:1035] T 837daa4ba2f1429b8cc5ae163707a343 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:18.890515 32270 consensus_queue.cc:1035] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:18.893249 32485 consensus_queue.cc:1035] T 07c25335b5e44744a67404fe1cd227b2 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:18.900802 22223 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:18.906630 32576 tablet.cc:1722] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:18.906901 32577 tablet.cc:1722] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:18.909701 32441 tablet.cc:1722] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:18.912819 22226 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=e5095259076a4a338b35330408063c71] alter complete (version 3)
I20240502 07:59:18.982316 31420 consensus_queue.cc:1035] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:18.988451 31770 consensus_queue.cc:1035] T 31fb10d4041c43b2b8a0f78866a53278 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:19.015518 22225 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:19.015722 22225 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:19.018131 22225 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet cc8e7771b0bd41b38b740266a7559a57
I20240502 07:59:19.019228 24930 tablet_service.cc:1514] Processing DeleteTablet for tablet cc8e7771b0bd41b38b740266a7559a57 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:19 UTC) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:19.019793 22352 tablet_service.cc:1514] Processing DeleteTablet for tablet cc8e7771b0bd41b38b740266a7559a57 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:19 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:19.020112 32629 tablet_replica.cc:330] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:19.020308 32629 raft_consensus.cc:2229] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:19.020457 32629 raft_consensus.cc:2258] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:19.020720 25193 tablet_service.cc:1514] Processing DeleteTablet for tablet cc8e7771b0bd41b38b740266a7559a57 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:19 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:19.021072 32630 tablet_replica.cc:330] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:19.021217 32629 ts_tablet_manager.cc:1875] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:19.021261 32630 raft_consensus.cc:2229] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:19.021493 32630 raft_consensus.cc:2258] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:19.021787 32628 tablet_replica.cc:330] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:19.021962 32628 raft_consensus.cc:2229] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:19.022181 32628 raft_consensus.cc:2258] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:19.022812 32628 ts_tablet_manager.cc:1875] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:19.022828 32630 ts_tablet_manager.cc:1875] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:19.024994 32628 ts_tablet_manager.cc:1888] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:19.025106 32628 log.cc:1196] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/cc8e7771b0bd41b38b740266a7559a57
I20240502 07:59:19.025455 32628 ts_tablet_manager.cc:1909] T cc8e7771b0bd41b38b740266a7559a57 P 1aecef81733c47b3b468504934357231: Deleting consensus metadata
I20240502 07:59:19.025518 32629 ts_tablet_manager.cc:1888] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:19.025611 32629 log.cc:1196] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/cc8e7771b0bd41b38b740266a7559a57
I20240502 07:59:19.025849 32630 ts_tablet_manager.cc:1888] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:19.025928 32629 ts_tablet_manager.cc:1909] T cc8e7771b0bd41b38b740266a7559a57 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:59:19.026005 32630 log.cc:1196] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/cc8e7771b0bd41b38b740266a7559a57
I20240502 07:59:19.026367 32630 ts_tablet_manager.cc:1909] T cc8e7771b0bd41b38b740266a7559a57 P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:59:19.026325 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet cc8e7771b0bd41b38b740266a7559a57 (table rebalancer_dynamic_table_new_name [id=e5095259076a4a338b35330408063c71]) successfully deleted
I20240502 07:59:19.026752 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet cc8e7771b0bd41b38b740266a7559a57 (table rebalancer_dynamic_table_new_name [id=e5095259076a4a338b35330408063c71]) successfully deleted
I20240502 07:59:19.027145 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet cc8e7771b0bd41b38b740266a7559a57 (table rebalancer_dynamic_table_new_name [id=e5095259076a4a338b35330408063c71]) successfully deleted
I20240502 07:59:19.042574 22352 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:19.042902 24930 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:19.044577 24800 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:19.051862 22614 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 07:59:19.059902 25193 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:19.071175 22225 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:19.079113 22743 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:19.080777 25063 tablet_service.cc:1467] Processing CreateTablet for tablet b13bad8f8cbe4a7b96f79ee10cb10172 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=e36e7eb676904ddeb7ed96bd9796a3bd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:19.081095 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b13bad8f8cbe4a7b96f79ee10cb10172. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.081594 22614 tablet_service.cc:1467] Processing CreateTablet for tablet b13bad8f8cbe4a7b96f79ee10cb10172 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=e36e7eb676904ddeb7ed96bd9796a3bd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:19.081902 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b13bad8f8cbe4a7b96f79ee10cb10172. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.084079 24930 tablet_service.cc:1467] Processing CreateTablet for tablet d1f2e047d45643ae9d091de1998931ed (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=e36e7eb676904ddeb7ed96bd9796a3bd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:19.084209 32643 tablet_bootstrap.cc:492] T b13bad8f8cbe4a7b96f79ee10cb10172 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:19.084375 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1f2e047d45643ae9d091de1998931ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.084515 22613 tablet_service.cc:1467] Processing CreateTablet for tablet d1f2e047d45643ae9d091de1998931ed (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=e36e7eb676904ddeb7ed96bd9796a3bd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:19.084744 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1f2e047d45643ae9d091de1998931ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.085160 32643 tablet_bootstrap.cc:654] T b13bad8f8cbe4a7b96f79ee10cb10172 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:19.086796 32645 tablet_bootstrap.cc:492] T d1f2e047d45643ae9d091de1998931ed P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:19.086745 24800 tablet_service.cc:1467] Processing CreateTablet for tablet eebad8d185b74842b576bddc1a7c7b0d (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=e36e7eb676904ddeb7ed96bd9796a3bd]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:19.087006 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eebad8d185b74842b576bddc1a7c7b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.088826 32645 tablet_bootstrap.cc:654] T d1f2e047d45643ae9d091de1998931ed P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:19.090795 32642 tablet_bootstrap.cc:492] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:19.091864 32642 tablet_bootstrap.cc:654] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:19.092835 32643 tablet_bootstrap.cc:492] T b13bad8f8cbe4a7b96f79ee10cb10172 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:19.092906 32643 ts_tablet_manager.cc:1367] T b13bad8f8cbe4a7b96f79ee10cb10172 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20240502 07:59:19.093272 32643 raft_consensus.cc:348] T b13bad8f8cbe4a7b96f79ee10cb10172 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.093415 32643 raft_consensus.cc:374] T b13bad8f8cbe4a7b96f79ee10cb10172 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:19.093452 32643 raft_consensus.cc:729] T b13bad8f8cbe4a7b96f79ee10cb10172 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.093571 32643 consensus_queue.cc:260] T b13bad8f8cbe4a7b96f79ee10cb10172 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.093870 32643 ts_tablet_manager.cc:1398] T b13bad8f8cbe4a7b96f79ee10cb10172 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:19.094054 32643 tablet_bootstrap.cc:492] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:19.095162 22481 tablet_service.cc:1467] Processing CreateTablet for tablet b13bad8f8cbe4a7b96f79ee10cb10172 (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=e36e7eb676904ddeb7ed96bd9796a3bd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:19.095163 22352 tablet_service.cc:1467] Processing CreateTablet for tablet eebad8d185b74842b576bddc1a7c7b0d (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=e36e7eb676904ddeb7ed96bd9796a3bd]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:19.095448 22481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b13bad8f8cbe4a7b96f79ee10cb10172. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.095489 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eebad8d185b74842b576bddc1a7c7b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.095528 32643 tablet_bootstrap.cc:654] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:19.096989 22482 tablet_service.cc:1467] Processing CreateTablet for tablet eebad8d185b74842b576bddc1a7c7b0d (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=e36e7eb676904ddeb7ed96bd9796a3bd]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:19.097206 32642 tablet_bootstrap.cc:492] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:19.097304 32642 ts_tablet_manager.cc:1367] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:19.097411 22483 tablet_service.cc:1467] Processing CreateTablet for tablet d1f2e047d45643ae9d091de1998931ed (DEFAULT_TABLE table=rebalancer_extra_table_56 [id=e36e7eb676904ddeb7ed96bd9796a3bd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:19.097618 22483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1f2e047d45643ae9d091de1998931ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.097828 32642 raft_consensus.cc:348] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.097939 32642 raft_consensus.cc:374] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:19.097970 32642 raft_consensus.cc:729] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.098079 32642 consensus_queue.cc:260] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.099778 32642 ts_tablet_manager.cc:1398] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:19.099988 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eebad8d185b74842b576bddc1a7c7b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.100533 32645 tablet_bootstrap.cc:492] T d1f2e047d45643ae9d091de1998931ed P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:19.100613 32645 ts_tablet_manager.cc:1367] T d1f2e047d45643ae9d091de1998931ed P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.001s
I20240502 07:59:19.100984 32646 tablet_bootstrap.cc:492] T eebad8d185b74842b576bddc1a7c7b0d P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:19.101027 32645 raft_consensus.cc:348] T d1f2e047d45643ae9d091de1998931ed P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.102170 32645 raft_consensus.cc:374] T d1f2e047d45643ae9d091de1998931ed P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:19.103612 32645 raft_consensus.cc:729] T d1f2e047d45643ae9d091de1998931ed P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.103797 32645 consensus_queue.cc:260] T d1f2e047d45643ae9d091de1998931ed P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.104941 32645 ts_tablet_manager.cc:1398] T d1f2e047d45643ae9d091de1998931ed P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20240502 07:59:19.105269 32650 tablet_bootstrap.cc:492] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:19.106316 32650 tablet_bootstrap.cc:654] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:19.120271 22225 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:19.122370 32650 tablet_bootstrap.cc:492] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:19.122488 32650 ts_tablet_manager.cc:1367] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 07:59:19.122977 32650 raft_consensus.cc:348] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.123116 32650 raft_consensus.cc:374] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:19.123152 32650 raft_consensus.cc:729] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.123273 32650 consensus_queue.cc:260] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.123594 32650 ts_tablet_manager.cc:1398] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:19.123996 32389 raft_consensus.cc:482] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:19.124078 32389 raft_consensus.cc:504] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.124686 32389 leader_election.cc:288] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:19.125257 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eebad8d185b74842b576bddc1a7c7b0d" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
W20240502 07:59:19.126346 22305 leader_election.cc:341] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): Illegal state: must be running to vote when last-logged opid is not known
I20240502 07:59:19.127600 22481 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:19.129364 25063 tablet_service.cc:1467] Processing CreateTablet for tablet bc2da9aa8c794fb581132cb59ae72fa7 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9d2639f697024d6d9c87950ed5a63c5e]), partition=
I20240502 07:59:19.129616 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc2da9aa8c794fb581132cb59ae72fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.130015 24930 tablet_service.cc:1467] Processing CreateTablet for tablet bc2da9aa8c794fb581132cb59ae72fa7 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9d2639f697024d6d9c87950ed5a63c5e]), partition=
I20240502 07:59:19.130223 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc2da9aa8c794fb581132cb59ae72fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.131043 25193 tablet_service.cc:1467] Processing CreateTablet for tablet bc2da9aa8c794fb581132cb59ae72fa7 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9d2639f697024d6d9c87950ed5a63c5e]), partition=
I20240502 07:59:19.131156 24819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eebad8d185b74842b576bddc1a7c7b0d" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:19.131309 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc2da9aa8c794fb581132cb59ae72fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.132114 32642 tablet_bootstrap.cc:492] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:19.132509 32645 tablet_bootstrap.cc:492] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:19.133165 32642 tablet_bootstrap.cc:654] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
W20240502 07:59:19.134574 22307 leader_election.cc:341] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Illegal state: must be running to vote when last-logged opid is not known
I20240502 07:59:19.134657 22307 leader_election.cc:302] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4; no voters: 4b4e7eeb7f3f42119c0363a79febc54b, fabddb16c85944648a710ec52fc7244c
I20240502 07:59:19.135720 32645 tablet_bootstrap.cc:654] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:19.136548 32389 raft_consensus.cc:2726] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 07:59:19.138456 32654 tablet_bootstrap.cc:492] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:19.141868 32646 tablet_bootstrap.cc:654] T eebad8d185b74842b576bddc1a7c7b0d P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:19.143191 32654 tablet_bootstrap.cc:654] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:19.145772 32654 tablet_bootstrap.cc:492] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:19.145853 32652 tablet_bootstrap.cc:492] T d1f2e047d45643ae9d091de1998931ed P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:19.145973 32654 ts_tablet_manager.cc:1367] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:59:19.146672 25063 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:19.147886 32643 tablet_bootstrap.cc:492] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:19.147991 32643 ts_tablet_manager.cc:1367] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.054s user 0.001s sys 0.001s
I20240502 07:59:19.148150 32654 raft_consensus.cc:348] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:19.148531 32643 raft_consensus.cc:348] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.148653 32643 raft_consensus.cc:374] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:19.148695 32643 raft_consensus.cc:729] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.148725 32654 raft_consensus.cc:374] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:19.148844 32654 raft_consensus.cc:729] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.148829 32643 consensus_queue.cc:260] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.149127 32643 ts_tablet_manager.cc:1398] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 07:59:19.149030 32654 consensus_queue.cc:260] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:19.152086 32654 ts_tablet_manager.cc:1398] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 07:59:19.155113 32652 tablet_bootstrap.cc:654] T d1f2e047d45643ae9d091de1998931ed P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:19.157660 32642 tablet_bootstrap.cc:492] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:19.157769 32642 ts_tablet_manager.cc:1367] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.002s
I20240502 07:59:19.158321 32642 raft_consensus.cc:348] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:19.158459 32642 raft_consensus.cc:374] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:19.158499 32642 raft_consensus.cc:729] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.158614 32642 consensus_queue.cc:260] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:19.158915 32642 ts_tablet_manager.cc:1398] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:19.165387 32645 tablet_bootstrap.cc:492] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:19.165498 32645 ts_tablet_manager.cc:1367] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20240502 07:59:19.165951 32645 raft_consensus.cc:348] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:19.166054 32645 raft_consensus.cc:374] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:19.166079 32645 raft_consensus.cc:729] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.166222 32645 consensus_queue.cc:260] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:19.166486 32645 ts_tablet_manager.cc:1398] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:19.170064 32646 tablet_bootstrap.cc:492] T eebad8d185b74842b576bddc1a7c7b0d P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:19.170189 32646 ts_tablet_manager.cc:1367] T eebad8d185b74842b576bddc1a7c7b0d P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.069s user 0.003s sys 0.000s
I20240502 07:59:19.170701 32646 raft_consensus.cc:348] T eebad8d185b74842b576bddc1a7c7b0d P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.170825 32646 raft_consensus.cc:374] T eebad8d185b74842b576bddc1a7c7b0d P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:19.170871 32646 raft_consensus.cc:729] T eebad8d185b74842b576bddc1a7c7b0d P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.171020 32646 consensus_queue.cc:260] T eebad8d185b74842b576bddc1a7c7b0d P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.171326 32646 ts_tablet_manager.cc:1398] T eebad8d185b74842b576bddc1a7c7b0d P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:19.189968 32652 tablet_bootstrap.cc:492] T d1f2e047d45643ae9d091de1998931ed P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:19.190097 32652 ts_tablet_manager.cc:1367] T d1f2e047d45643ae9d091de1998931ed P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.044s user 0.003s sys 0.000s
I20240502 07:59:19.190665 32652 raft_consensus.cc:348] T d1f2e047d45643ae9d091de1998931ed P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.190793 32652 raft_consensus.cc:374] T d1f2e047d45643ae9d091de1998931ed P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:19.190834 32652 raft_consensus.cc:729] T d1f2e047d45643ae9d091de1998931ed P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.190959 32652 consensus_queue.cc:260] T d1f2e047d45643ae9d091de1998931ed P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.191293 32652 ts_tablet_manager.cc:1398] T d1f2e047d45643ae9d091de1998931ed P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:19.191464 32652 tablet_bootstrap.cc:492] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:19.192615 32652 tablet_bootstrap.cc:654] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:19.198067 32652 tablet_bootstrap.cc:492] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:19.198186 32652 ts_tablet_manager.cc:1367] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:19.198649 32652 raft_consensus.cc:348] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.198770 32652 raft_consensus.cc:374] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:19.198812 32652 raft_consensus.cc:729] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.198962 32652 consensus_queue.cc:260] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.199280 32652 ts_tablet_manager.cc:1398] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:19.199580 32652 tablet_bootstrap.cc:492] T b13bad8f8cbe4a7b96f79ee10cb10172 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:19.200525 32652 tablet_bootstrap.cc:654] T b13bad8f8cbe4a7b96f79ee10cb10172 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:19.202061 32652 tablet_bootstrap.cc:492] T b13bad8f8cbe4a7b96f79ee10cb10172 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:19.202198 32652 ts_tablet_manager.cc:1367] T b13bad8f8cbe4a7b96f79ee10cb10172 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:19.202817 32652 raft_consensus.cc:348] T b13bad8f8cbe4a7b96f79ee10cb10172 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.202952 32652 raft_consensus.cc:374] T b13bad8f8cbe4a7b96f79ee10cb10172 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:19.203002 32652 raft_consensus.cc:729] T b13bad8f8cbe4a7b96f79ee10cb10172 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.203166 32652 consensus_queue.cc:260] T b13bad8f8cbe4a7b96f79ee10cb10172 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.203578 32652 ts_tablet_manager.cc:1398] T b13bad8f8cbe4a7b96f79ee10cb10172 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:19.259469 24952 consensus_queue.cc:237] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:19.261109 25084 raft_consensus.cc:1261] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:19.261420 24888 consensus_peers.cc:485] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 -> Peer f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): Couldn't send request to peer f00f3a1876444ad2bf9c919068ab8014. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 461098a3f8e64fe89a8603f78dc36dd2. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.261660 22501 raft_consensus.cc:1261] T 461098a3f8e64fe89a8603f78dc36dd2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.261763 32477 consensus_queue.cc:1035] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.262341 32477 consensus_queue.cc:1035] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.264305 32477 raft_consensus.cc:2932] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:19.264709 22501 raft_consensus.cc:2932] T 461098a3f8e64fe89a8603f78dc36dd2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:19.266455 25084 raft_consensus.cc:2932] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:19.269158 22230 catalog_manager.cc:5484] T 461098a3f8e64fe89a8603f78dc36dd2 P fabddb16c85944648a710ec52fc7244c reported cstate change: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:19.290426 22632 consensus_queue.cc:237] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.291759 24952 raft_consensus.cc:1261] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.291916 22763 raft_consensus.cc:1261] T fe087f9b25c84749b7cfba34800da679 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.292119 31410 consensus_queue.cc:1035] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 07:59:19.292284 22567 consensus_peers.cc:485] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f -> Peer fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): Couldn't send request to peer fabddb16c85944648a710ec52fc7244c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fe087f9b25c84749b7cfba34800da679. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.292518 31410 consensus_queue.cc:1035] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.294562 32099 raft_consensus.cc:2932] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.296573 24952 raft_consensus.cc:2932] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.296623 22763 raft_consensus.cc:2932] T fe087f9b25c84749b7cfba34800da679 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.297544 22226 catalog_manager.cc:5484] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.305081 32660 raft_consensus.cc:482] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:19.305205 32660 raft_consensus.cc:504] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.305707 32660 leader_election.cc:288] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:19.306006 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b13bad8f8cbe4a7b96f79ee10cb10172" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:19.306142 22632 raft_consensus.cc:2454] T b13bad8f8cbe4a7b96f79ee10cb10172 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:19.306142 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b13bad8f8cbe4a7b96f79ee10cb10172" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:19.306289 22501 raft_consensus.cc:2454] T b13bad8f8cbe4a7b96f79ee10cb10172 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:19.306562 25020 leader_election.cc:302] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:19.306849 32660 raft_consensus.cc:2781] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:19.306900 32660 raft_consensus.cc:482] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:19.306927 32660 raft_consensus.cc:3037] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:19.307885 32660 raft_consensus.cc:504] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.308234 32660 leader_election.cc:288] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:19.308543 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b13bad8f8cbe4a7b96f79ee10cb10172" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:19.308543 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b13bad8f8cbe4a7b96f79ee10cb10172" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:19.308648 22501 raft_consensus.cc:3037] T b13bad8f8cbe4a7b96f79ee10cb10172 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:19.308656 22632 raft_consensus.cc:3037] T b13bad8f8cbe4a7b96f79ee10cb10172 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:19.309551 22501 raft_consensus.cc:2454] T b13bad8f8cbe4a7b96f79ee10cb10172 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:19.309564 22632 raft_consensus.cc:2454] T b13bad8f8cbe4a7b96f79ee10cb10172 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:19.309827 25017 leader_election.cc:302] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:19.310151 32660 raft_consensus.cc:2781] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:19.310237 32660 raft_consensus.cc:686] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:19.310388 32660 consensus_queue.cc:237] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.312999 22230 catalog_manager.cc:5484] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.323645 22763 consensus_queue.cc:237] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:19.325054 22632 raft_consensus.cc:1261] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20240502 07:59:19.325139 22701 consensus_peers.cc:485] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 -> Peer 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): Couldn't send request to peer 43930dee0cd5498fa8e357a34dd71432. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 895f9a31785d4558acd3b64904a645ca. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.325392 25084 raft_consensus.cc:1261] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:19.325464 32718 consensus_queue.cc:1035] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:19.325806 32536 consensus_queue.cc:1035] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:19.327595 32718 raft_consensus.cc:2932] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:19.328042 25084 raft_consensus.cc:2932] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:19.329968 22632 raft_consensus.cc:2932] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:19.331530 22230 catalog_manager.cc:5484] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: config changed from index 5 to 6, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:19.352254 24952 consensus_queue.cc:237] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:19.353696 22632 raft_consensus.cc:1261] T d63f00a2d007442781d58fb938bed480 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:19.353835 24887 consensus_peers.cc:485] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d63f00a2d007442781d58fb938bed480. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.354154 32270 consensus_queue.cc:1035] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.354311 22370 raft_consensus.cc:1261] T d63f00a2d007442781d58fb938bed480 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.354688 32270 consensus_queue.cc:1035] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.358143 32270 raft_consensus.cc:2932] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:19.358456 22632 raft_consensus.cc:2932] T d63f00a2d007442781d58fb938bed480 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:19.360096 22370 raft_consensus.cc:2932] T d63f00a2d007442781d58fb938bed480 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:19.362259 22225 catalog_manager.cc:5484] T d63f00a2d007442781d58fb938bed480 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index -1 to 2, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:19.384490 24819 consensus_queue.cc:237] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:19.386265 24952 raft_consensus.cc:1261] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer 4b4e7eeb7f3f42119c0363a79febc54b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:19.386423 24757 consensus_peers.cc:485] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b -> Peer f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): Couldn't send request to peer f00f3a1876444ad2bf9c919068ab8014. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5034380b5c0846faa1494140d67203cc. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.386627 32756 consensus_queue.cc:1035] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.387264 25084 raft_consensus.cc:1261] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 4b4e7eeb7f3f42119c0363a79febc54b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.387630 32756 consensus_queue.cc:1035] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.389454 32178 raft_consensus.cc:2932] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:19.389863 25084 raft_consensus.cc:2932] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:19.392753 22230 catalog_manager.cc:5484] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:19.393605 24952 raft_consensus.cc:2932] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:19.394572 31325 raft_consensus.cc:482] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:19.394670 31325 raft_consensus.cc:504] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:19.395179 31325 leader_election.cc:288] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:19.395614 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc2da9aa8c794fb581132cb59ae72fa7" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:19.395746 25084 raft_consensus.cc:2454] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:19.396122 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc2da9aa8c794fb581132cb59ae72fa7" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:19.396148 25148 leader_election.cc:302] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:19.396225 24952 raft_consensus.cc:2454] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:19.396306 31325 raft_consensus.cc:2781] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:19.396350 31325 raft_consensus.cc:482] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:19.396399 31325 raft_consensus.cc:3037] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:19.397372 31325 raft_consensus.cc:504] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:19.397763 31325 leader_election.cc:288] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:19.398280 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc2da9aa8c794fb581132cb59ae72fa7" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:19.398391 24952 raft_consensus.cc:3037] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:19.399225 24952 raft_consensus.cc:2454] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:19.399474 25151 leader_election.cc:302] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:19.399631 31325 raft_consensus.cc:2781] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:19.399718 31325 raft_consensus.cc:686] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:19.399838 31325 consensus_queue.cc:237] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:19.401160 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc2da9aa8c794fb581132cb59ae72fa7" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:19.401270 25084 raft_consensus.cc:3037] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:19.402225 25084 raft_consensus.cc:2454] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:19.402706 22227 catalog_manager.cc:5484] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:19.415752 22370 consensus_queue.cc:237] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.417093 24952 raft_consensus.cc:1261] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.417289 25084 raft_consensus.cc:1261] T ff6bc97390164a37bcdaafd651c888cf P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.417743 308 consensus_queue.cc:1035] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 07:59:19.418910 22305 consensus_peers.cc:485] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 -> Peer fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): Couldn't send request to peer fabddb16c85944648a710ec52fc7244c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff6bc97390164a37bcdaafd651c888cf. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.419654 308 consensus_queue.cc:1035] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.421520 32389 raft_consensus.cc:2932] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.421942 25084 raft_consensus.cc:2932] T ff6bc97390164a37bcdaafd651c888cf P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.423442 24952 raft_consensus.cc:2932] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.424865 22227 catalog_manager.cc:5484] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.448516 22370 consensus_queue.cc:237] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:19.450101 22632 raft_consensus.cc:1261] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:19.450127 24952 raft_consensus.cc:1261] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:19.450557 311 consensus_queue.cc:1035] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:19.450742 32390 consensus_queue.cc:1035] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20240502 07:59:19.451076 22308 consensus_peers.cc:485] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): Couldn't send request to peer 43930dee0cd5498fa8e357a34dd71432. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 45925e9713cb4b748efde06f77cf3702. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.452556 311 raft_consensus.cc:2932] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:19.452991 24952 raft_consensus.cc:2932] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:19.455215 22632 raft_consensus.cc:2932] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:19.457609 22230 catalog_manager.cc:5484] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index 5 to 6, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) added. New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:19.494315 22763 consensus_queue.cc:237] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:19.495802 24952 raft_consensus.cc:1261] T c5a30e1ebd7747eb911fba8437e42077 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:19.495915 22372 raft_consensus.cc:1261] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:19.496503 32718 consensus_queue.cc:1035] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20240502 07:59:19.496701 22700 consensus_peers.cc:485] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c5a30e1ebd7747eb911fba8437e42077. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.496716 31905 consensus_queue.cc:1035] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:19.498751 32536 raft_consensus.cc:2932] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:19.499233 22372 raft_consensus.cc:2932] T c5a30e1ebd7747eb911fba8437e42077 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:19.502955 22227 catalog_manager.cc:5484] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index 5 to 6, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:19.504182 24952 raft_consensus.cc:2932] T c5a30e1ebd7747eb911fba8437e42077 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:19.529302 25084 consensus_queue.cc:237] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:19.529675 32698 raft_consensus.cc:482] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:19.529775 32698 raft_consensus.cc:504] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.530138 32698 leader_election.cc:288] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:19.530947 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1f2e047d45643ae9d091de1998931ed" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:19.531075 24952 raft_consensus.cc:2454] T d1f2e047d45643ae9d091de1998931ed P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:19.531417 25213 raft_consensus.cc:1261] T 26b5df5842324628b2e9bb6dc1fc36f2 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.531562 22632 raft_consensus.cc:1261] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:19.531749 25020 consensus_peers.cc:485] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 -> Peer f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): Couldn't send request to peer f00f3a1876444ad2bf9c919068ab8014. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 26b5df5842324628b2e9bb6dc1fc36f2. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.531769 32529 raft_consensus.cc:482] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:19.531877 32529 raft_consensus.cc:504] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.532218 365 consensus_queue.cc:1035] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.532260 32529 leader_election.cc:288] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:19.533197 22570 leader_election.cc:302] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:19.533179 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eebad8d185b74842b576bddc1a7c7b0d" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:19.533310 22372 raft_consensus.cc:2454] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:19.533812 22439 leader_election.cc:302] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:19.533972 32529 raft_consensus.cc:2781] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:19.534040 32529 raft_consensus.cc:482] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:19.534094 32529 raft_consensus.cc:3037] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:19.534263 32660 consensus_queue.cc:1035] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.534962 32529 raft_consensus.cc:504] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.535326 32529 leader_election.cc:288] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:19.536227 24820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eebad8d185b74842b576bddc1a7c7b0d" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" is_pre_election: true
I20240502 07:59:19.536378 24820 raft_consensus.cc:2454] T eebad8d185b74842b576bddc1a7c7b0d P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:19.536414 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1f2e047d45643ae9d091de1998931ed" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:19.536528 22501 raft_consensus.cc:2454] T d1f2e047d45643ae9d091de1998931ed P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:19.536252 24819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eebad8d185b74842b576bddc1a7c7b0d" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4e7eeb7f3f42119c0363a79febc54b"
I20240502 07:59:19.536553 22225 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:19.536695 24819 raft_consensus.cc:3037] T eebad8d185b74842b576bddc1a7c7b0d P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:19.537725 24819 raft_consensus.cc:2454] T eebad8d185b74842b576bddc1a7c7b0d P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:19.537812 32698 raft_consensus.cc:2781] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:19.537876 32698 raft_consensus.cc:482] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:19.537917 32698 raft_consensus.cc:3037] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:19.538878 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eebad8d185b74842b576bddc1a7c7b0d" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:19.538892 32698 raft_consensus.cc:504] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.538985 22372 raft_consensus.cc:3037] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:19.539289 32698 leader_election.cc:288] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:19.539278 22438 leader_election.cc:302] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4e7eeb7f3f42119c0363a79febc54b, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:19.539431 32529 raft_consensus.cc:2781] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:19.539490 32529 raft_consensus.cc:686] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:59:19.539598 32529 consensus_queue.cc:237] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.539949 22372 raft_consensus.cc:2454] T eebad8d185b74842b576bddc1a7c7b0d P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:19.541678 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1f2e047d45643ae9d091de1998931ed" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:19.541800 24952 raft_consensus.cc:3037] T d1f2e047d45643ae9d091de1998931ed P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:19.542501 22227 catalog_manager.cc:5484] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.542729 24952 raft_consensus.cc:2454] T d1f2e047d45643ae9d091de1998931ed P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:19.543334 22570 leader_election.cc:302] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:19.544106 25084 raft_consensus.cc:1261] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.544770 31325 consensus_queue.cc:1035] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:19.545364 32660 raft_consensus.cc:2932] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:19.546664 24952 raft_consensus.cc:1261] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.547489 25213 raft_consensus.cc:2932] T 26b5df5842324628b2e9bb6dc1fc36f2 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:19.547466 22632 raft_consensus.cc:2932] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:19.547727 22227 catalog_manager.cc:5576] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733) does not have the latest schema for tablet bc2da9aa8c794fb581132cb59ae72fa7 (table rebalancer_dynamic_table [id=9d2639f697024d6d9c87950ed5a63c5e]). Expected version 1 got 0
I20240502 07:59:19.548151 31325 consensus_queue.cc:1035] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:19.548919 32099 raft_consensus.cc:2781] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:19.549002 32099 raft_consensus.cc:686] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:59:19.549137 32099 consensus_queue.cc:237] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.551574 22227 catalog_manager.cc:5576] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995) does not have the latest schema for tablet bc2da9aa8c794fb581132cb59ae72fa7 (table rebalancer_dynamic_table [id=9d2639f697024d6d9c87950ed5a63c5e]). Expected version 1 got 0
I20240502 07:59:19.554451 22227 catalog_manager.cc:5484] T 26b5df5842324628b2e9bb6dc1fc36f2 P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: config changed from index -1 to 2, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) added. New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:19.556432 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1f2e047d45643ae9d091de1998931ed" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:19.556557 22501 raft_consensus.cc:3037] T d1f2e047d45643ae9d091de1998931ed P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:19.557641 22501 raft_consensus.cc:2454] T d1f2e047d45643ae9d091de1998931ed P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:19.559980 22227 catalog_manager.cc:5484] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.561143 22225 catalog_manager.cc:5576] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733) does not have the latest schema for tablet bc2da9aa8c794fb581132cb59ae72fa7 (table rebalancer_dynamic_table [id=9d2639f697024d6d9c87950ed5a63c5e]). Expected version 1 got 0
I20240502 07:59:19.561635 381 tablet.cc:1722] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:19.561637 376 tablet.cc:1722] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:19.562021 376 tablet.cc:1620] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:19.563948 381 tablet.cc:1620] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:19.565543 22226 catalog_manager.cc:5576] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995) does not have the latest schema for tablet bc2da9aa8c794fb581132cb59ae72fa7 (table rebalancer_dynamic_table [id=9d2639f697024d6d9c87950ed5a63c5e]). Expected version 1 got 0
I20240502 07:59:19.569713 22227 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=9d2639f697024d6d9c87950ed5a63c5e] alter complete (version 1)
I20240502 07:59:19.570101 383 tablet.cc:1722] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:19.570607 383 tablet.cc:1620] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:19.594398 22632 consensus_queue.cc:237] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.595665 24819 raft_consensus.cc:1261] T f033e8bbac8445f3af64faa67c818987 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.596040 32698 consensus_queue.cc:1035] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.596140 24952 raft_consensus.cc:1261] T f033e8bbac8445f3af64faa67c818987 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:19.596201 22567 consensus_peers.cc:485] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f -> Peer fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): Couldn't send request to peer fabddb16c85944648a710ec52fc7244c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f033e8bbac8445f3af64faa67c818987. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.596477 32698 consensus_queue.cc:1035] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.598053 32099 raft_consensus.cc:2932] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.598493 24952 raft_consensus.cc:2932] T f033e8bbac8445f3af64faa67c818987 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.599779 24819 raft_consensus.cc:2932] T f033e8bbac8445f3af64faa67c818987 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.601032 22225 catalog_manager.cc:5484] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.628062 22632 consensus_queue.cc:237] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:19.629897 22763 raft_consensus.cc:1261] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20240502 07:59:19.629910 22569 consensus_peers.cc:485] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 72964b1c935441ca840a46ff79921cc3. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.629981 25213 raft_consensus.cc:1261] T 72964b1c935441ca840a46ff79921cc3 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:19.630367 32099 consensus_queue.cc:1035] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:19.630542 31410 consensus_queue.cc:1035] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:19.632287 32099 raft_consensus.cc:2932] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:19.632679 22763 raft_consensus.cc:2932] T 72964b1c935441ca840a46ff79921cc3 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:19.636392 22226 catalog_manager.cc:5484] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index 5 to 6, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:19.636899 25213 raft_consensus.cc:2932] T 72964b1c935441ca840a46ff79921cc3 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:19.658921 24819 consensus_queue.cc:237] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:19.660609 25213 raft_consensus.cc:1261] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 4b4e7eeb7f3f42119c0363a79febc54b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:19.660773 24757 consensus_peers.cc:485] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b -> Peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): Couldn't send request to peer b533a7d8d97943609c6e28e3e36b78a4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 05911d56b0344eb8a53aba5f4bf0170c. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.661010 32756 consensus_queue.cc:1035] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.661715 25084 raft_consensus.cc:1261] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 4b4e7eeb7f3f42119c0363a79febc54b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.662135 32756 consensus_queue.cc:1035] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.664002 32460 raft_consensus.cc:2932] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:19.666842 25213 raft_consensus.cc:2932] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:19.668604 22226 catalog_manager.cc:5484] T 05911d56b0344eb8a53aba5f4bf0170c P 4b4e7eeb7f3f42119c0363a79febc54b reported cstate change: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New cstate: current_term: 1 leader_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:19.670091 25084 raft_consensus.cc:2932] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:19.673494 22226 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:19.679939 376 tablet.cc:1722] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:19.680264 376 tablet.cc:1620] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:19.680475 383 tablet.cc:1722] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:19.680783 383 tablet.cc:1620] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:19.685038 381 tablet.cc:1722] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:19.686295 22227 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=9d2639f697024d6d9c87950ed5a63c5e] alter complete (version 2)
I20240502 07:59:19.686543 381 tablet.cc:1620] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:19.690637 372 consensus_queue.cc:1035] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:19.695185 32292 consensus_queue.cc:1035] T b13bad8f8cbe4a7b96f79ee10cb10172 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:19.702947 22632 consensus_queue.cc:237] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:19.704380 22763 raft_consensus.cc:1261] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
W20240502 07:59:19.704726 22567 consensus_peers.cc:485] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): Couldn't send request to peer 227eec4b5f014d349661f6e0b39b9055. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2241a741bf85433b9fea512e9c530416. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.704780 25213 raft_consensus.cc:1261] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20240502 07:59:19.705161 415 consensus_queue.cc:1035] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:19.705309 32698 consensus_queue.cc:1035] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:19.707507 32698 raft_consensus.cc:2932] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:19.707881 22763 raft_consensus.cc:2932] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:19.710239 25213 raft_consensus.cc:2932] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:19.711937 22227 catalog_manager.cc:5484] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: config changed from index 5 to 6, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) added. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:19.733852 22763 consensus_queue.cc:237] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:19.735365 25213 raft_consensus.cc:1261] T 5242aa2d2d9c477994026edb0d42051c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.735525 24952 raft_consensus.cc:1261] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:19.735662 22701 consensus_peers.cc:485] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 -> Peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): Couldn't send request to peer e51b10af49484aaa8f0267aebc1bf27f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5242aa2d2d9c477994026edb0d42051c. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.735926 348 consensus_queue.cc:1035] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.736286 348 consensus_queue.cc:1035] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.738452 32718 raft_consensus.cc:2932] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:19.738914 25213 raft_consensus.cc:2932] T 5242aa2d2d9c477994026edb0d42051c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:19.740597 24952 raft_consensus.cc:2932] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:19.742007 22227 catalog_manager.cc:5484] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:19.766523 25213 consensus_queue.cc:237] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:19.767905 22501 raft_consensus.cc:1261] T 89889b6805684edeb79afd3fa3a55dea P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.767856 22763 raft_consensus.cc:1261] T 89889b6805684edeb79afd3fa3a55dea P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:19.768002 25150 consensus_peers.cc:485] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 89889b6805684edeb79afd3fa3a55dea. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.768306 492 consensus_queue.cc:1035] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.768537 32103 consensus_queue.cc:1035] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.770455 495 ts_tablet_manager.cc:902] T fe087f9b25c84749b7cfba34800da679 P fabddb16c85944648a710ec52fc7244c: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:19.770730 492 raft_consensus.cc:2932] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:19.770973 495 tablet_copy_client.cc:308] T fe087f9b25c84749b7cfba34800da679 P fabddb16c85944648a710ec52fc7244c: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:59:19.771201 22764 raft_consensus.cc:2932] T 89889b6805684edeb79afd3fa3a55dea P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:19.772560 22653 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet fe087f9b25c84749b7cfba34800da679 from peer fabddb16c85944648a710ec52fc7244c ({username='slave'} at 127.15.129.130:46959)
I20240502 07:59:19.772680 22653 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet fe087f9b25c84749b7cfba34800da679 from peer fabddb16c85944648a710ec52fc7244c at {username='slave'} at 127.15.129.130:46959: session id = fabddb16c85944648a710ec52fc7244c-fe087f9b25c84749b7cfba34800da679
I20240502 07:59:19.773600 22653 tablet_copy_source_session.cc:205] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:19.773535 22501 raft_consensus.cc:2932] T 89889b6805684edeb79afd3fa3a55dea P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:19.774407 22226 catalog_manager.cc:5484] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: config changed from index -1 to 2, VOTER 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) added. New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:19.775537 495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe087f9b25c84749b7cfba34800da679. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.777745 495 tablet_copy_client.cc:783] T fe087f9b25c84749b7cfba34800da679 P fabddb16c85944648a710ec52fc7244c: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:19.777913 495 tablet_copy_client.cc:647] T fe087f9b25c84749b7cfba34800da679 P fabddb16c85944648a710ec52fc7244c: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:19.781339 495 tablet_copy_client.cc:523] T fe087f9b25c84749b7cfba34800da679 P fabddb16c85944648a710ec52fc7244c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:19.782580 495 tablet_bootstrap.cc:492] T fe087f9b25c84749b7cfba34800da679 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:19.785563 495 tablet_bootstrap.cc:492] T fe087f9b25c84749b7cfba34800da679 P fabddb16c85944648a710ec52fc7244c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:19.785856 495 tablet_bootstrap.cc:492] T fe087f9b25c84749b7cfba34800da679 P fabddb16c85944648a710ec52fc7244c: Bootstrap complete.
I20240502 07:59:19.786021 495 ts_tablet_manager.cc:1367] T fe087f9b25c84749b7cfba34800da679 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20240502 07:59:19.786432 495 raft_consensus.cc:348] T fe087f9b25c84749b7cfba34800da679 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.786530 495 raft_consensus.cc:729] T fe087f9b25c84749b7cfba34800da679 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.786660 495 consensus_queue.cc:260] T fe087f9b25c84749b7cfba34800da679 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.786945 495 ts_tablet_manager.cc:1398] T fe087f9b25c84749b7cfba34800da679 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:19.787693 22653 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session fabddb16c85944648a710ec52fc7244c-fe087f9b25c84749b7cfba34800da679 received from {username='slave'} at 127.15.129.130:46959
I20240502 07:59:19.787781 22653 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session fabddb16c85944648a710ec52fc7244c-fe087f9b25c84749b7cfba34800da679 on tablet fe087f9b25c84749b7cfba34800da679 with peer fabddb16c85944648a710ec52fc7244c
I20240502 07:59:19.790087 22226 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:19.795604 376 tablet.cc:1722] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:19.796056 381 tablet.cc:1722] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:19.796841 383 tablet.cc:1722] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:19.801697 22225 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=9d2639f697024d6d9c87950ed5a63c5e] alter complete (version 3)
I20240502 07:59:19.801967 516 ts_tablet_manager.cc:902] T 461098a3f8e64fe89a8603f78dc36dd2 P f00f3a1876444ad2bf9c919068ab8014: Initiating tablet copy from peer 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:19.803470 516 tablet_copy_client.cc:308] T 461098a3f8e64fe89a8603f78dc36dd2 P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.134:35995
I20240502 07:59:19.804718 24972 tablet_copy_service.cc:140] P 1aecef81733c47b3b468504934357231: Received BeginTabletCopySession request for tablet 461098a3f8e64fe89a8603f78dc36dd2 from peer f00f3a1876444ad2bf9c919068ab8014 ({username='slave'} at 127.15.129.132:59331)
I20240502 07:59:19.804822 24972 tablet_copy_service.cc:161] P 1aecef81733c47b3b468504934357231: Beginning new tablet copy session on tablet 461098a3f8e64fe89a8603f78dc36dd2 from peer f00f3a1876444ad2bf9c919068ab8014 at {username='slave'} at 127.15.129.132:59331: session id = f00f3a1876444ad2bf9c919068ab8014-461098a3f8e64fe89a8603f78dc36dd2
I20240502 07:59:19.806110 24972 tablet_copy_source_session.cc:205] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:19.806718 22763 consensus_queue.cc:237] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:19.806885 516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 461098a3f8e64fe89a8603f78dc36dd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.808077 22501 raft_consensus.cc:1261] T 038a695352ca4da58afaeb63a7785866 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.808099 25213 raft_consensus.cc:1261] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:19.808372 22701 consensus_peers.cc:485] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 -> Peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): Couldn't send request to peer b533a7d8d97943609c6e28e3e36b78a4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 038a695352ca4da58afaeb63a7785866. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.808477 31905 consensus_queue.cc:1035] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.808773 348 consensus_queue.cc:1035] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.809268 516 tablet_copy_client.cc:783] T 461098a3f8e64fe89a8603f78dc36dd2 P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:19.809443 516 tablet_copy_client.cc:647] T 461098a3f8e64fe89a8603f78dc36dd2 P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:19.810464 516 tablet_copy_client.cc:523] T 461098a3f8e64fe89a8603f78dc36dd2 P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:19.810817 348 raft_consensus.cc:2932] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:19.811079 25213 raft_consensus.cc:2932] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:19.812134 516 tablet_bootstrap.cc:492] T 461098a3f8e64fe89a8603f78dc36dd2 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:19.813485 22501 raft_consensus.cc:2932] T 038a695352ca4da58afaeb63a7785866 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:19.814087 22227 catalog_manager.cc:5484] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:19.817145 516 tablet_bootstrap.cc:492] T 461098a3f8e64fe89a8603f78dc36dd2 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:19.817422 516 tablet_bootstrap.cc:492] T 461098a3f8e64fe89a8603f78dc36dd2 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap complete.
I20240502 07:59:19.817592 516 ts_tablet_manager.cc:1367] T 461098a3f8e64fe89a8603f78dc36dd2 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20240502 07:59:19.818022 516 raft_consensus.cc:348] T 461098a3f8e64fe89a8603f78dc36dd2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:19.818131 516 raft_consensus.cc:729] T 461098a3f8e64fe89a8603f78dc36dd2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.818262 516 consensus_queue.cc:260] T 461098a3f8e64fe89a8603f78dc36dd2 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:19.818554 516 ts_tablet_manager.cc:1398] T 461098a3f8e64fe89a8603f78dc36dd2 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:19.819617 24972 tablet_copy_service.cc:342] P 1aecef81733c47b3b468504934357231: Request end of tablet copy session f00f3a1876444ad2bf9c919068ab8014-461098a3f8e64fe89a8603f78dc36dd2 received from {username='slave'} at 127.15.129.132:59331
I20240502 07:59:19.819697 24972 tablet_copy_service.cc:433] P 1aecef81733c47b3b468504934357231: ending tablet copy session f00f3a1876444ad2bf9c919068ab8014-461098a3f8e64fe89a8603f78dc36dd2 on tablet 461098a3f8e64fe89a8603f78dc36dd2 with peer f00f3a1876444ad2bf9c919068ab8014
I20240502 07:59:19.836846 25213 consensus_queue.cc:237] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:19.838155 22763 raft_consensus.cc:1261] T 2ea24049cec547b0a6cb85be6033eb3d P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.838203 24819 raft_consensus.cc:1261] T 2ea24049cec547b0a6cb85be6033eb3d P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:19.838295 25151 consensus_peers.cc:485] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 -> Peer 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): Couldn't send request to peer 1aecef81733c47b3b468504934357231. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2ea24049cec547b0a6cb85be6033eb3d. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.838559 31325 consensus_queue.cc:1035] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.838711 32103 consensus_queue.cc:1035] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.839805 516 ts_tablet_manager.cc:902] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014: Initiating tablet copy from peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:19.840265 516 tablet_copy_client.cc:308] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.133:41661
I20240502 07:59:19.840659 24841 tablet_copy_service.cc:140] P 4b4e7eeb7f3f42119c0363a79febc54b: Received BeginTabletCopySession request for tablet 5034380b5c0846faa1494140d67203cc from peer f00f3a1876444ad2bf9c919068ab8014 ({username='slave'} at 127.15.129.132:52299)
I20240502 07:59:19.840751 24841 tablet_copy_service.cc:161] P 4b4e7eeb7f3f42119c0363a79febc54b: Beginning new tablet copy session on tablet 5034380b5c0846faa1494140d67203cc from peer f00f3a1876444ad2bf9c919068ab8014 at {username='slave'} at 127.15.129.132:52299: session id = f00f3a1876444ad2bf9c919068ab8014-5034380b5c0846faa1494140d67203cc
I20240502 07:59:19.841816 24841 tablet_copy_source_session.cc:205] T 5034380b5c0846faa1494140d67203cc P 4b4e7eeb7f3f42119c0363a79febc54b: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:19.842669 516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5034380b5c0846faa1494140d67203cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.842965 32103 raft_consensus.cc:2932] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:19.844403 22764 raft_consensus.cc:2932] T 2ea24049cec547b0a6cb85be6033eb3d P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:19.844681 516 tablet_copy_client.cc:783] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:19.844825 516 tablet_copy_client.cc:647] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:19.846349 22226 catalog_manager.cc:5484] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: config changed from index -1 to 2, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:19.846577 24821 raft_consensus.cc:2932] T 2ea24049cec547b0a6cb85be6033eb3d P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:19.850086 516 tablet_copy_client.cc:523] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:19.851300 516 tablet_bootstrap.cc:492] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:19.854804 516 tablet_bootstrap.cc:492] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:19.855041 516 tablet_bootstrap.cc:492] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014: Bootstrap complete.
I20240502 07:59:19.855202 516 ts_tablet_manager.cc:1367] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:19.855846 516 raft_consensus.cc:348] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:19.855958 516 raft_consensus.cc:729] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.856103 516 consensus_queue.cc:260] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:19.856487 516 ts_tablet_manager.cc:1398] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:19.857462 24841 tablet_copy_service.cc:342] P 4b4e7eeb7f3f42119c0363a79febc54b: Request end of tablet copy session f00f3a1876444ad2bf9c919068ab8014-5034380b5c0846faa1494140d67203cc received from {username='slave'} at 127.15.129.132:52299
I20240502 07:59:19.857545 24841 tablet_copy_service.cc:433] P 4b4e7eeb7f3f42119c0363a79febc54b: ending tablet copy session f00f3a1876444ad2bf9c919068ab8014-5034380b5c0846faa1494140d67203cc on tablet 5034380b5c0846faa1494140d67203cc with peer f00f3a1876444ad2bf9c919068ab8014
I20240502 07:59:19.871271 25213 consensus_queue.cc:237] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:19.872572 25084 raft_consensus.cc:1261] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:19.872931 25151 consensus_peers.cc:485] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 -> Peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): Couldn't send request to peer e51b10af49484aaa8f0267aebc1bf27f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a7da69fb73f94ce0b8c24599c8ae4d15. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.873025 24952 raft_consensus.cc:1261] T a7da69fb73f94ce0b8c24599c8ae4d15 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.873019 500 consensus_queue.cc:1035] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.873471 500 consensus_queue.cc:1035] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.875087 31325 raft_consensus.cc:2932] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:19.875468 24952 raft_consensus.cc:2932] T a7da69fb73f94ce0b8c24599c8ae4d15 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:19.877204 25084 raft_consensus.cc:2932] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:19.878350 22226 catalog_manager.cc:5484] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: config changed from index -1 to 2, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) added. New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:19.899693 24952 consensus_queue.cc:237] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
W20240502 07:59:19.901180 24885 consensus_peers.cc:485] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 -> Peer fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): Couldn't send request to peer fabddb16c85944648a710ec52fc7244c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: caf2dd9c96b94566b216692a9370f16e. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.901218 22632 raft_consensus.cc:1261] T caf2dd9c96b94566b216692a9370f16e P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.901230 24821 raft_consensus.cc:1261] T caf2dd9c96b94566b216692a9370f16e P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.901697 584 consensus_queue.cc:1035] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.901870 32270 consensus_queue.cc:1035] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.904421 587 ts_tablet_manager.cc:902] T 895f9a31785d4558acd3b64904a645ca P 43930dee0cd5498fa8e357a34dd71432: Initiating tablet copy from peer f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:19.905082 587 tablet_copy_client.cc:308] T 895f9a31785d4558acd3b64904a645ca P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.132:41673
I20240502 07:59:19.905184 22225 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:19.905403 22225 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:19.905670 22785 tablet_copy_service.cc:140] P f00f3a1876444ad2bf9c919068ab8014: Received BeginTabletCopySession request for tablet 895f9a31785d4558acd3b64904a645ca from peer 43930dee0cd5498fa8e357a34dd71432 ({username='slave'} at 127.15.129.136:55029)
I20240502 07:59:19.905792 22785 tablet_copy_service.cc:161] P f00f3a1876444ad2bf9c919068ab8014: Beginning new tablet copy session on tablet 895f9a31785d4558acd3b64904a645ca from peer 43930dee0cd5498fa8e357a34dd71432 at {username='slave'} at 127.15.129.136:55029: session id = 43930dee0cd5498fa8e357a34dd71432-895f9a31785d4558acd3b64904a645ca
I20240502 07:59:19.906795 22785 tablet_copy_source_session.cc:205] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:19.907739 587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 895f9a31785d4558acd3b64904a645ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.907994 22225 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet bc2da9aa8c794fb581132cb59ae72fa7
I20240502 07:59:19.908243 592 ts_tablet_manager.cc:902] T 45925e9713cb4b748efde06f77cf3702 P 43930dee0cd5498fa8e357a34dd71432: Initiating tablet copy from peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:19.908819 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet bc2da9aa8c794fb581132cb59ae72fa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:19 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:19.909272 24930 tablet_service.cc:1514] Processing DeleteTablet for tablet bc2da9aa8c794fb581132cb59ae72fa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:19 UTC) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:19.909296 592 tablet_copy_client.cc:308] T 45925e9713cb4b748efde06f77cf3702 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.129:38535
I20240502 07:59:19.909436 25193 tablet_service.cc:1514] Processing DeleteTablet for tablet bc2da9aa8c794fb581132cb59ae72fa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:19 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:19.909708 22392 tablet_copy_service.cc:140] P b533a7d8d97943609c6e28e3e36b78a4: Received BeginTabletCopySession request for tablet 45925e9713cb4b748efde06f77cf3702 from peer 43930dee0cd5498fa8e357a34dd71432 ({username='slave'} at 127.15.129.136:51007)
I20240502 07:59:19.909798 22392 tablet_copy_service.cc:161] P b533a7d8d97943609c6e28e3e36b78a4: Beginning new tablet copy session on tablet 45925e9713cb4b748efde06f77cf3702 from peer 43930dee0cd5498fa8e357a34dd71432 at {username='slave'} at 127.15.129.136:51007: session id = 43930dee0cd5498fa8e357a34dd71432-45925e9713cb4b748efde06f77cf3702
I20240502 07:59:19.910126 587 tablet_copy_client.cc:783] T 895f9a31785d4558acd3b64904a645ca P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:19.910379 587 tablet_copy_client.cc:647] T 895f9a31785d4558acd3b64904a645ca P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:19.910728 22392 tablet_copy_source_session.cc:205] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:19.910952 595 tablet_replica.cc:330] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:19.912098 592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45925e9713cb4b748efde06f77cf3702. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.913486 598 tablet_replica.cc:330] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:19.913653 598 raft_consensus.cc:2229] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:19.913764 598 raft_consensus.cc:2258] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:19.914008 592 tablet_copy_client.cc:783] T 45925e9713cb4b748efde06f77cf3702 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:19.914175 592 tablet_copy_client.cc:647] T 45925e9713cb4b748efde06f77cf3702 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:19.914440 598 ts_tablet_manager.cc:1875] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:19.914587 587 tablet_copy_client.cc:523] T 895f9a31785d4558acd3b64904a645ca P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:19.915760 587 tablet_bootstrap.cc:492] T 895f9a31785d4558acd3b64904a645ca P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:19.917289 598 ts_tablet_manager.cc:1888] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:19.917431 598 log.cc:1196] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/bc2da9aa8c794fb581132cb59ae72fa7
I20240502 07:59:19.917694 598 ts_tablet_manager.cc:1909] T bc2da9aa8c794fb581132cb59ae72fa7 P 1aecef81733c47b3b468504934357231: Deleting consensus metadata
I20240502 07:59:19.917852 592 tablet_copy_client.cc:523] T 45925e9713cb4b748efde06f77cf3702 P 43930dee0cd5498fa8e357a34dd71432: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:19.918180 32270 raft_consensus.cc:2932] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.919083 592 tablet_bootstrap.cc:492] T 45925e9713cb4b748efde06f77cf3702 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:19.920454 24821 raft_consensus.cc:2932] T caf2dd9c96b94566b216692a9370f16e P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.922027 22632 raft_consensus.cc:2932] T caf2dd9c96b94566b216692a9370f16e P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.911139 595 raft_consensus.cc:2229] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:19.924299 595 raft_consensus.cc:2258] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:19.924829 22230 catalog_manager.cc:5484] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 reported cstate change: config changed from index -1 to 2, VOTER fabddb16c85944648a710ec52fc7244c (127.15.129.130) added. New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:19.926458 595 ts_tablet_manager.cc:1875] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:19.929713 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet bc2da9aa8c794fb581132cb59ae72fa7 (table rebalancer_dynamic_table_new_name [id=9d2639f697024d6d9c87950ed5a63c5e]) successfully deleted
I20240502 07:59:19.933840 595 ts_tablet_manager.cc:1888] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:19.933933 595 log.cc:1196] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/bc2da9aa8c794fb581132cb59ae72fa7
I20240502 07:59:19.934229 595 ts_tablet_manager.cc:1909] T bc2da9aa8c794fb581132cb59ae72fa7 P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:59:19.934716 599 tablet_replica.cc:330] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:19.934886 599 raft_consensus.cc:2229] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:19.935125 599 raft_consensus.cc:2258] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:19.936136 599 ts_tablet_manager.cc:1875] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:19.936198 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet bc2da9aa8c794fb581132cb59ae72fa7 (table rebalancer_dynamic_table_new_name [id=9d2639f697024d6d9c87950ed5a63c5e]) successfully deleted
I20240502 07:59:19.938544 599 ts_tablet_manager.cc:1888] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:19.938656 599 log.cc:1196] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/bc2da9aa8c794fb581132cb59ae72fa7
I20240502 07:59:19.939079 599 ts_tablet_manager.cc:1909] T bc2da9aa8c794fb581132cb59ae72fa7 P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:59:19.939815 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet bc2da9aa8c794fb581132cb59ae72fa7 (table rebalancer_dynamic_table_new_name [id=9d2639f697024d6d9c87950ed5a63c5e]) successfully deleted
I20240502 07:59:19.949957 614 ts_tablet_manager.cc:902] T d63f00a2d007442781d58fb938bed480 P 4b4e7eeb7f3f42119c0363a79febc54b: Initiating tablet copy from peer 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:19.950558 614 tablet_copy_client.cc:308] T d63f00a2d007442781d58fb938bed480 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.134:35995
I20240502 07:59:19.951052 24972 tablet_copy_service.cc:140] P 1aecef81733c47b3b468504934357231: Received BeginTabletCopySession request for tablet d63f00a2d007442781d58fb938bed480 from peer 4b4e7eeb7f3f42119c0363a79febc54b ({username='slave'} at 127.15.129.133:58103)
I20240502 07:59:19.951159 24972 tablet_copy_service.cc:161] P 1aecef81733c47b3b468504934357231: Beginning new tablet copy session on tablet d63f00a2d007442781d58fb938bed480 from peer 4b4e7eeb7f3f42119c0363a79febc54b at {username='slave'} at 127.15.129.133:58103: session id = 4b4e7eeb7f3f42119c0363a79febc54b-d63f00a2d007442781d58fb938bed480
I20240502 07:59:19.952090 24972 tablet_copy_source_session.cc:205] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:19.953014 614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d63f00a2d007442781d58fb938bed480. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.956179 614 tablet_copy_client.cc:783] T d63f00a2d007442781d58fb938bed480 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:19.956712 25084 consensus_queue.cc:237] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:19.958478 24952 raft_consensus.cc:1261] T 5d342c368796426c81bede7ddc23937b P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:19.958889 25213 raft_consensus.cc:1261] T 5d342c368796426c81bede7ddc23937b P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20240502 07:59:19.959300 25020 consensus_peers.cc:485] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 -> Peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): Couldn't send request to peer b533a7d8d97943609c6e28e3e36b78a4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5d342c368796426c81bede7ddc23937b. This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:19.959503 365 consensus_queue.cc:1035] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.959684 372 consensus_queue.cc:1035] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:19.962284 614 tablet_copy_client.cc:647] T d63f00a2d007442781d58fb938bed480 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:19.963483 614 tablet_copy_client.cc:523] T d63f00a2d007442781d58fb938bed480 P 4b4e7eeb7f3f42119c0363a79febc54b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:19.964867 614 tablet_bootstrap.cc:492] T d63f00a2d007442781d58fb938bed480 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:19.965520 372 raft_consensus.cc:2932] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:19.968623 614 tablet_bootstrap.cc:492] T d63f00a2d007442781d58fb938bed480 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:19.968886 614 tablet_bootstrap.cc:492] T d63f00a2d007442781d58fb938bed480 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap complete.
I20240502 07:59:19.969038 614 ts_tablet_manager.cc:1367] T d63f00a2d007442781d58fb938bed480 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20240502 07:59:19.969035 587 tablet_bootstrap.cc:492] T 895f9a31785d4558acd3b64904a645ca P 43930dee0cd5498fa8e357a34dd71432: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:19.968899 22227 catalog_manager.cc:5484] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:19.969416 587 tablet_bootstrap.cc:492] T 895f9a31785d4558acd3b64904a645ca P 43930dee0cd5498fa8e357a34dd71432: Bootstrap complete.
I20240502 07:59:19.969570 587 ts_tablet_manager.cc:1367] T 895f9a31785d4558acd3b64904a645ca P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.054s user 0.026s sys 0.003s
I20240502 07:59:19.969520 614 raft_consensus.cc:348] T d63f00a2d007442781d58fb938bed480 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:19.969620 614 raft_consensus.cc:729] T d63f00a2d007442781d58fb938bed480 P 4b4e7eeb7f3f42119c0363a79febc54b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.969738 614 consensus_queue.cc:260] T d63f00a2d007442781d58fb938bed480 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:19.970048 614 ts_tablet_manager.cc:1398] T d63f00a2d007442781d58fb938bed480 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:19.970024 587 raft_consensus.cc:348] T 895f9a31785d4558acd3b64904a645ca P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:19.970196 587 raft_consensus.cc:729] T 895f9a31785d4558acd3b64904a645ca P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.970433 587 consensus_queue.cc:260] T 895f9a31785d4558acd3b64904a645ca P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:19.970733 587 ts_tablet_manager.cc:1398] T 895f9a31785d4558acd3b64904a645ca P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:19.972254 24972 tablet_copy_service.cc:342] P 1aecef81733c47b3b468504934357231: Request end of tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-d63f00a2d007442781d58fb938bed480 received from {username='slave'} at 127.15.129.133:58103
I20240502 07:59:19.972339 24972 tablet_copy_service.cc:433] P 1aecef81733c47b3b468504934357231: ending tablet copy session 4b4e7eeb7f3f42119c0363a79febc54b-d63f00a2d007442781d58fb938bed480 on tablet d63f00a2d007442781d58fb938bed480 with peer 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:19.972415 24952 raft_consensus.cc:2932] T 5d342c368796426c81bede7ddc23937b P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:19.972992 22785 tablet_copy_service.cc:342] P f00f3a1876444ad2bf9c919068ab8014: Request end of tablet copy session 43930dee0cd5498fa8e357a34dd71432-895f9a31785d4558acd3b64904a645ca received from {username='slave'} at 127.15.129.136:55029
I20240502 07:59:19.973083 22785 tablet_copy_service.cc:433] P f00f3a1876444ad2bf9c919068ab8014: ending tablet copy session 43930dee0cd5498fa8e357a34dd71432-895f9a31785d4558acd3b64904a645ca on tablet 895f9a31785d4558acd3b64904a645ca with peer 43930dee0cd5498fa8e357a34dd71432
I20240502 07:59:19.973722 592 tablet_bootstrap.cc:492] T 45925e9713cb4b748efde06f77cf3702 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=689 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:19.974023 592 tablet_bootstrap.cc:492] T 45925e9713cb4b748efde06f77cf3702 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap complete.
I20240502 07:59:19.974189 592 ts_tablet_manager.cc:1367] T 45925e9713cb4b748efde06f77cf3702 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.055s user 0.031s sys 0.000s
I20240502 07:59:19.974503 25213 raft_consensus.cc:2932] T 5d342c368796426c81bede7ddc23937b P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:19.974575 592 raft_consensus.cc:348] T 45925e9713cb4b748efde06f77cf3702 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:19.974687 592 raft_consensus.cc:729] T 45925e9713cb4b748efde06f77cf3702 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.974798 592 consensus_queue.cc:260] T 45925e9713cb4b748efde06f77cf3702 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:19.975132 592 ts_tablet_manager.cc:1398] T 45925e9713cb4b748efde06f77cf3702 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:19.975672 22392 tablet_copy_service.cc:342] P b533a7d8d97943609c6e28e3e36b78a4: Request end of tablet copy session 43930dee0cd5498fa8e357a34dd71432-45925e9713cb4b748efde06f77cf3702 received from {username='slave'} at 127.15.129.136:51007
I20240502 07:59:19.975754 22392 tablet_copy_service.cc:433] P b533a7d8d97943609c6e28e3e36b78a4: ending tablet copy session 43930dee0cd5498fa8e357a34dd71432-45925e9713cb4b748efde06f77cf3702 on tablet 45925e9713cb4b748efde06f77cf3702 with peer 43930dee0cd5498fa8e357a34dd71432
I20240502 07:59:19.983892 495 ts_tablet_manager.cc:902] T f033e8bbac8445f3af64faa67c818987 P fabddb16c85944648a710ec52fc7244c: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:19.984340 495 tablet_copy_client.cc:308] T f033e8bbac8445f3af64faa67c818987 P fabddb16c85944648a710ec52fc7244c: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:59:19.984763 22653 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet f033e8bbac8445f3af64faa67c818987 from peer fabddb16c85944648a710ec52fc7244c ({username='slave'} at 127.15.129.130:46959)
I20240502 07:59:19.984870 22653 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet f033e8bbac8445f3af64faa67c818987 from peer fabddb16c85944648a710ec52fc7244c at {username='slave'} at 127.15.129.130:46959: session id = fabddb16c85944648a710ec52fc7244c-f033e8bbac8445f3af64faa67c818987
I20240502 07:59:19.985849 22653 tablet_copy_source_session.cc:205] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:19.986699 495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f033e8bbac8445f3af64faa67c818987. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:19.988485 495 tablet_copy_client.cc:783] T f033e8bbac8445f3af64faa67c818987 P fabddb16c85944648a710ec52fc7244c: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:19.988617 495 tablet_copy_client.cc:647] T f033e8bbac8445f3af64faa67c818987 P fabddb16c85944648a710ec52fc7244c: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:19.990100 495 tablet_copy_client.cc:523] T f033e8bbac8445f3af64faa67c818987 P fabddb16c85944648a710ec52fc7244c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:19.991276 495 tablet_bootstrap.cc:492] T f033e8bbac8445f3af64faa67c818987 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:19.995021 495 tablet_bootstrap.cc:492] T f033e8bbac8445f3af64faa67c818987 P fabddb16c85944648a710ec52fc7244c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:19.995262 495 tablet_bootstrap.cc:492] T f033e8bbac8445f3af64faa67c818987 P fabddb16c85944648a710ec52fc7244c: Bootstrap complete.
I20240502 07:59:19.995398 495 ts_tablet_manager.cc:1367] T f033e8bbac8445f3af64faa67c818987 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 07:59:19.995787 495 raft_consensus.cc:348] T f033e8bbac8445f3af64faa67c818987 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.995899 495 raft_consensus.cc:729] T f033e8bbac8445f3af64faa67c818987 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:19.996012 495 consensus_queue.cc:260] T f033e8bbac8445f3af64faa67c818987 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:19.996342 495 ts_tablet_manager.cc:1398] T f033e8bbac8445f3af64faa67c818987 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:19.996902 22653 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session fabddb16c85944648a710ec52fc7244c-f033e8bbac8445f3af64faa67c818987 received from {username='slave'} at 127.15.129.130:46959
I20240502 07:59:19.996976 22653 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session fabddb16c85944648a710ec52fc7244c-f033e8bbac8445f3af64faa67c818987 on tablet f033e8bbac8445f3af64faa67c818987 with peer fabddb16c85944648a710ec52fc7244c
I20240502 07:59:20.004089 32698 consensus_queue.cc:1035] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:20.007066 415 consensus_queue.cc:1035] T d1f2e047d45643ae9d091de1998931ed P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:20.009541 22225 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:20.015503 495 ts_tablet_manager.cc:902] T ff6bc97390164a37bcdaafd651c888cf P fabddb16c85944648a710ec52fc7244c: Initiating tablet copy from peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:20.015947 22613 tablet_service.cc:1467] Processing CreateTablet for tablet 16764c31484a43a4a0b705ce536bde0a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a30a987f1fcf4907a3a5ed5f97997f7c]), partition=
I20240502 07:59:20.015995 495 tablet_copy_client.cc:308] T ff6bc97390164a37bcdaafd651c888cf P fabddb16c85944648a710ec52fc7244c: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.129:38535
I20240502 07:59:20.016083 24930 tablet_service.cc:1467] Processing CreateTablet for tablet 16764c31484a43a4a0b705ce536bde0a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a30a987f1fcf4907a3a5ed5f97997f7c]), partition=
I20240502 07:59:20.016216 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16764c31484a43a4a0b705ce536bde0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.016330 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16764c31484a43a4a0b705ce536bde0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.016419 22392 tablet_copy_service.cc:140] P b533a7d8d97943609c6e28e3e36b78a4: Received BeginTabletCopySession request for tablet ff6bc97390164a37bcdaafd651c888cf from peer fabddb16c85944648a710ec52fc7244c ({username='slave'} at 127.15.129.130:50039)
I20240502 07:59:20.016501 22392 tablet_copy_service.cc:161] P b533a7d8d97943609c6e28e3e36b78a4: Beginning new tablet copy session on tablet ff6bc97390164a37bcdaafd651c888cf from peer fabddb16c85944648a710ec52fc7244c at {username='slave'} at 127.15.129.130:50039: session id = fabddb16c85944648a710ec52fc7244c-ff6bc97390164a37bcdaafd651c888cf
I20240502 07:59:20.017453 22392 tablet_copy_source_session.cc:205] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:20.018321 495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff6bc97390164a37bcdaafd651c888cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.018723 655 tablet_bootstrap.cc:492] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:20.018781 656 tablet_bootstrap.cc:492] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:20.019757 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 16764c31484a43a4a0b705ce536bde0a (DEFAULT_TABLE table=rebalancer_dynamic_table [id=a30a987f1fcf4907a3a5ed5f97997f7c]), partition=
I20240502 07:59:20.019968 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16764c31484a43a4a0b705ce536bde0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.020548 495 tablet_copy_client.cc:783] T ff6bc97390164a37bcdaafd651c888cf P fabddb16c85944648a710ec52fc7244c: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:20.020682 495 tablet_copy_client.cc:647] T ff6bc97390164a37bcdaafd651c888cf P fabddb16c85944648a710ec52fc7244c: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:20.021462 656 tablet_bootstrap.cc:654] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:20.021822 495 tablet_copy_client.cc:523] T ff6bc97390164a37bcdaafd651c888cf P fabddb16c85944648a710ec52fc7244c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:20.023121 495 tablet_bootstrap.cc:492] T ff6bc97390164a37bcdaafd651c888cf P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:20.023734 655 tablet_bootstrap.cc:654] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:20.024482 516 ts_tablet_manager.cc:902] T 26b5df5842324628b2e9bb6dc1fc36f2 P f00f3a1876444ad2bf9c919068ab8014: Initiating tablet copy from peer 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:20.025462 516 tablet_copy_client.cc:308] T 26b5df5842324628b2e9bb6dc1fc36f2 P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.135:42733
I20240502 07:59:20.025774 656 tablet_bootstrap.cc:492] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:20.025880 656 ts_tablet_manager.cc:1367] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240502 07:59:20.026381 656 raft_consensus.cc:348] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.027113 495 tablet_bootstrap.cc:492] T ff6bc97390164a37bcdaafd651c888cf P fabddb16c85944648a710ec52fc7244c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:20.027114 656 raft_consensus.cc:374] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:20.027290 656 raft_consensus.cc:729] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.027511 495 tablet_bootstrap.cc:492] T ff6bc97390164a37bcdaafd651c888cf P fabddb16c85944648a710ec52fc7244c: Bootstrap complete.
I20240502 07:59:20.027462 656 consensus_queue.cc:260] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.027766 657 tablet_bootstrap.cc:492] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:20.027842 656 ts_tablet_manager.cc:1398] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:59:20.027734 495 ts_tablet_manager.cc:1367] T ff6bc97390164a37bcdaafd651c888cf P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.005s
I20240502 07:59:20.028882 25104 tablet_copy_service.cc:140] P 227eec4b5f014d349661f6e0b39b9055: Received BeginTabletCopySession request for tablet 26b5df5842324628b2e9bb6dc1fc36f2 from peer f00f3a1876444ad2bf9c919068ab8014 ({username='slave'} at 127.15.129.132:43799)
I20240502 07:59:20.028998 25104 tablet_copy_service.cc:161] P 227eec4b5f014d349661f6e0b39b9055: Beginning new tablet copy session on tablet 26b5df5842324628b2e9bb6dc1fc36f2 from peer f00f3a1876444ad2bf9c919068ab8014 at {username='slave'} at 127.15.129.132:43799: session id = f00f3a1876444ad2bf9c919068ab8014-26b5df5842324628b2e9bb6dc1fc36f2
I20240502 07:59:20.029361 495 raft_consensus.cc:348] T ff6bc97390164a37bcdaafd651c888cf P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:20.029467 495 raft_consensus.cc:729] T ff6bc97390164a37bcdaafd651c888cf P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.029613 495 consensus_queue.cc:260] T ff6bc97390164a37bcdaafd651c888cf P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:20.029978 495 ts_tablet_manager.cc:1398] T ff6bc97390164a37bcdaafd651c888cf P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:20.030051 25104 tablet_copy_source_session.cc:205] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:20.030856 22392 tablet_copy_service.cc:342] P b533a7d8d97943609c6e28e3e36b78a4: Request end of tablet copy session fabddb16c85944648a710ec52fc7244c-ff6bc97390164a37bcdaafd651c888cf received from {username='slave'} at 127.15.129.130:50039
I20240502 07:59:20.030931 22392 tablet_copy_service.cc:433] P b533a7d8d97943609c6e28e3e36b78a4: ending tablet copy session fabddb16c85944648a710ec52fc7244c-ff6bc97390164a37bcdaafd651c888cf on tablet ff6bc97390164a37bcdaafd651c888cf with peer fabddb16c85944648a710ec52fc7244c
I20240502 07:59:20.032297 31770 consensus_queue.cc:1035] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:20.034610 516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26b5df5842324628b2e9bb6dc1fc36f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.036798 516 tablet_copy_client.cc:783] T 26b5df5842324628b2e9bb6dc1fc36f2 P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:20.036947 516 tablet_copy_client.cc:647] T 26b5df5842324628b2e9bb6dc1fc36f2 P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:20.042454 31770 consensus_queue.cc:1035] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:20.043586 516 tablet_copy_client.cc:523] T 26b5df5842324628b2e9bb6dc1fc36f2 P f00f3a1876444ad2bf9c919068ab8014: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:20.044462 657 tablet_bootstrap.cc:654] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:20.045102 516 tablet_bootstrap.cc:492] T 26b5df5842324628b2e9bb6dc1fc36f2 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:20.045929 655 tablet_bootstrap.cc:492] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:20.046089 655 ts_tablet_manager.cc:1367] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20240502 07:59:20.046823 655 raft_consensus.cc:348] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.046993 655 raft_consensus.cc:374] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:20.047062 655 raft_consensus.cc:729] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.047225 655 consensus_queue.cc:260] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.048370 655 ts_tablet_manager.cc:1398] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:20.048904 657 tablet_bootstrap.cc:492] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:20.048921 516 tablet_bootstrap.cc:492] T 26b5df5842324628b2e9bb6dc1fc36f2 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:20.049134 657 ts_tablet_manager.cc:1367] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20240502 07:59:20.049258 516 tablet_bootstrap.cc:492] T 26b5df5842324628b2e9bb6dc1fc36f2 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap complete.
I20240502 07:59:20.049451 516 ts_tablet_manager.cc:1367] T 26b5df5842324628b2e9bb6dc1fc36f2 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240502 07:59:20.049799 657 raft_consensus.cc:348] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.049854 516 raft_consensus.cc:348] T 26b5df5842324628b2e9bb6dc1fc36f2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.049993 657 raft_consensus.cc:374] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:20.050015 516 raft_consensus.cc:729] T 26b5df5842324628b2e9bb6dc1fc36f2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.050159 657 raft_consensus.cc:729] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.050256 516 consensus_queue.cc:260] T 26b5df5842324628b2e9bb6dc1fc36f2 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.050433 657 consensus_queue.cc:260] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.050565 516 ts_tablet_manager.cc:1398] T 26b5df5842324628b2e9bb6dc1fc36f2 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:20.050941 657 ts_tablet_manager.cc:1398] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:20.051255 25104 tablet_copy_service.cc:342] P 227eec4b5f014d349661f6e0b39b9055: Request end of tablet copy session f00f3a1876444ad2bf9c919068ab8014-26b5df5842324628b2e9bb6dc1fc36f2 received from {username='slave'} at 127.15.129.132:43799
I20240502 07:59:20.051343 25104 tablet_copy_service.cc:433] P 227eec4b5f014d349661f6e0b39b9055: ending tablet copy session f00f3a1876444ad2bf9c919068ab8014-26b5df5842324628b2e9bb6dc1fc36f2 on tablet 26b5df5842324628b2e9bb6dc1fc36f2 with peer f00f3a1876444ad2bf9c919068ab8014
I20240502 07:59:20.063982 22224 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:20.072285 24800 tablet_service.cc:1467] Processing CreateTablet for tablet a511340ea9e649c690b11afb9a35835f (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=4884748f5d5945a7bebc525ceffda105]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:20.072559 24800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a511340ea9e649c690b11afb9a35835f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.073212 25062 tablet_service.cc:1467] Processing CreateTablet for tablet 5ee489f17d80406d8fd5c2909612f456 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=4884748f5d5945a7bebc525ceffda105]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:20.073472 25062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ee489f17d80406d8fd5c2909612f456. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.074322 24799 tablet_service.cc:1467] Processing CreateTablet for tablet 5ee489f17d80406d8fd5c2909612f456 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=4884748f5d5945a7bebc525ceffda105]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:20.074594 24799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ee489f17d80406d8fd5c2909612f456. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.075649 698 tablet_bootstrap.cc:492] T 5ee489f17d80406d8fd5c2909612f456 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:20.073179 25063 tablet_service.cc:1467] Processing CreateTablet for tablet a511340ea9e649c690b11afb9a35835f (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=4884748f5d5945a7bebc525ceffda105]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:20.076380 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a511340ea9e649c690b11afb9a35835f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.077107 698 tablet_bootstrap.cc:654] T 5ee489f17d80406d8fd5c2909612f456 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:20.079550 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 497e7c78a2274cc1a8406cdab125e0d3 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=4884748f5d5945a7bebc525ceffda105]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:20.079629 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 5ee489f17d80406d8fd5c2909612f456 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=4884748f5d5945a7bebc525ceffda105]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:20.079818 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 497e7c78a2274cc1a8406cdab125e0d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.079746 22352 tablet_service.cc:1467] Processing CreateTablet for tablet a511340ea9e649c690b11afb9a35835f (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=4884748f5d5945a7bebc525ceffda105]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:20.079958 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ee489f17d80406d8fd5c2909612f456. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.080016 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a511340ea9e649c690b11afb9a35835f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.080269 25193 tablet_service.cc:1467] Processing CreateTablet for tablet 497e7c78a2274cc1a8406cdab125e0d3 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=4884748f5d5945a7bebc525ceffda105]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:20.080518 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 497e7c78a2274cc1a8406cdab125e0d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.081096 697 tablet_bootstrap.cc:492] T a511340ea9e649c690b11afb9a35835f P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:20.082538 22351 tablet_service.cc:1467] Processing CreateTablet for tablet 497e7c78a2274cc1a8406cdab125e0d3 (DEFAULT_TABLE table=rebalancer_extra_table_58 [id=4884748f5d5945a7bebc525ceffda105]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:20.082782 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 497e7c78a2274cc1a8406cdab125e0d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.082846 701 tablet_bootstrap.cc:492] T 497e7c78a2274cc1a8406cdab125e0d3 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:20.082892 657 tablet_bootstrap.cc:492] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:20.083174 697 tablet_bootstrap.cc:654] T a511340ea9e649c690b11afb9a35835f P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:20.083803 657 tablet_bootstrap.cc:654] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:20.083868 701 tablet_bootstrap.cc:654] T 497e7c78a2274cc1a8406cdab125e0d3 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:20.085587 698 tablet_bootstrap.cc:492] T 5ee489f17d80406d8fd5c2909612f456 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:20.086840 698 ts_tablet_manager.cc:1367] T 5ee489f17d80406d8fd5c2909612f456 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 07:59:20.087267 698 raft_consensus.cc:348] T 5ee489f17d80406d8fd5c2909612f456 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.087368 698 raft_consensus.cc:374] T 5ee489f17d80406d8fd5c2909612f456 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:20.087401 698 raft_consensus.cc:729] T 5ee489f17d80406d8fd5c2909612f456 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.087539 698 consensus_queue.cc:260] T 5ee489f17d80406d8fd5c2909612f456 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.087832 698 ts_tablet_manager.cc:1398] T 5ee489f17d80406d8fd5c2909612f456 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:20.096587 700 tablet_bootstrap.cc:492] T a511340ea9e649c690b11afb9a35835f P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:20.096691 698 tablet_bootstrap.cc:492] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:20.097535 700 tablet_bootstrap.cc:654] T a511340ea9e649c690b11afb9a35835f P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:20.098342 657 tablet_bootstrap.cc:492] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:20.098445 657 ts_tablet_manager.cc:1367] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20240502 07:59:20.098706 698 tablet_bootstrap.cc:654] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:20.099081 657 raft_consensus.cc:348] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.099208 657 raft_consensus.cc:374] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:20.099249 657 raft_consensus.cc:729] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.099402 657 consensus_queue.cc:260] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.099686 657 ts_tablet_manager.cc:1398] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:20.099835 657 tablet_bootstrap.cc:492] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:20.100260 698 tablet_bootstrap.cc:492] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:20.100334 698 ts_tablet_manager.cc:1367] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:20.100843 657 tablet_bootstrap.cc:654] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:20.101102 698 raft_consensus.cc:348] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:20.101384 698 raft_consensus.cc:374] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:20.101943 698 raft_consensus.cc:729] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.102083 698 consensus_queue.cc:260] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:20.102394 698 ts_tablet_manager.cc:1398] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:20.102747 697 tablet_bootstrap.cc:492] T a511340ea9e649c690b11afb9a35835f P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:20.102876 697 ts_tablet_manager.cc:1367] T a511340ea9e649c690b11afb9a35835f P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20240502 07:59:20.102931 657 tablet_bootstrap.cc:492] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:20.103008 657 ts_tablet_manager.cc:1367] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:20.103453 657 raft_consensus.cc:348] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.103560 657 raft_consensus.cc:374] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:20.103593 657 raft_consensus.cc:729] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.103718 657 consensus_queue.cc:260] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.103938 701 tablet_bootstrap.cc:492] T 497e7c78a2274cc1a8406cdab125e0d3 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:20.103981 657 ts_tablet_manager.cc:1398] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:20.104032 701 ts_tablet_manager.cc:1367] T 497e7c78a2274cc1a8406cdab125e0d3 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 07:59:20.103996 697 raft_consensus.cc:348] T a511340ea9e649c690b11afb9a35835f P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:20.104133 697 raft_consensus.cc:374] T a511340ea9e649c690b11afb9a35835f P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:20.104173 697 raft_consensus.cc:729] T a511340ea9e649c690b11afb9a35835f P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.104313 697 consensus_queue.cc:260] T a511340ea9e649c690b11afb9a35835f P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:20.104727 697 ts_tablet_manager.cc:1398] T a511340ea9e649c690b11afb9a35835f P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:20.105034 700 tablet_bootstrap.cc:492] T a511340ea9e649c690b11afb9a35835f P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:20.105176 697 tablet_bootstrap.cc:492] T 5ee489f17d80406d8fd5c2909612f456 P 4b4e7eeb7f3f42119c0363a79febc54b: Bootstrap starting.
I20240502 07:59:20.105175 700 ts_tablet_manager.cc:1367] T a511340ea9e649c690b11afb9a35835f P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 07:59:20.106688 697 tablet_bootstrap.cc:654] T 5ee489f17d80406d8fd5c2909612f456 P 4b4e7eeb7f3f42119c0363a79febc54b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:20.106624 701 raft_consensus.cc:348] T 497e7c78a2274cc1a8406cdab125e0d3 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.106735 701 raft_consensus.cc:374] T 497e7c78a2274cc1a8406cdab125e0d3 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:20.106801 701 raft_consensus.cc:729] T 497e7c78a2274cc1a8406cdab125e0d3 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.107194 700 raft_consensus.cc:348] T a511340ea9e649c690b11afb9a35835f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:20.107390 700 raft_consensus.cc:374] T a511340ea9e649c690b11afb9a35835f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:20.107474 700 raft_consensus.cc:729] T a511340ea9e649c690b11afb9a35835f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.107645 700 consensus_queue.cc:260] T a511340ea9e649c690b11afb9a35835f P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:20.107991 700 ts_tablet_manager.cc:1398] T a511340ea9e649c690b11afb9a35835f P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20240502 07:59:20.106953 701 consensus_queue.cc:260] T 497e7c78a2274cc1a8406cdab125e0d3 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.108340 700 tablet_bootstrap.cc:492] T 497e7c78a2274cc1a8406cdab125e0d3 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:20.108383 701 ts_tablet_manager.cc:1398] T 497e7c78a2274cc1a8406cdab125e0d3 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:20.109395 700 tablet_bootstrap.cc:654] T 497e7c78a2274cc1a8406cdab125e0d3 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:20.112326 700 tablet_bootstrap.cc:492] T 497e7c78a2274cc1a8406cdab125e0d3 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:20.112408 700 ts_tablet_manager.cc:1367] T 497e7c78a2274cc1a8406cdab125e0d3 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20240502 07:59:20.112848 700 raft_consensus.cc:348] T 497e7c78a2274cc1a8406cdab125e0d3 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.112955 700 raft_consensus.cc:374] T 497e7c78a2274cc1a8406cdab125e0d3 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:20.112995 700 raft_consensus.cc:729] T 497e7c78a2274cc1a8406cdab125e0d3 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.113121 700 consensus_queue.cc:260] T 497e7c78a2274cc1a8406cdab125e0d3 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.113206 697 tablet_bootstrap.cc:492] T 5ee489f17d80406d8fd5c2909612f456 P 4b4e7eeb7f3f42119c0363a79febc54b: No bootstrap required, opened a new log
I20240502 07:59:20.113278 697 ts_tablet_manager.cc:1367] T 5ee489f17d80406d8fd5c2909612f456 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:59:20.113416 700 ts_tablet_manager.cc:1398] T 497e7c78a2274cc1a8406cdab125e0d3 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:20.113760 697 raft_consensus.cc:348] T 5ee489f17d80406d8fd5c2909612f456 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.113893 697 raft_consensus.cc:374] T 5ee489f17d80406d8fd5c2909612f456 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:20.113948 697 raft_consensus.cc:729] T 5ee489f17d80406d8fd5c2909612f456 P 4b4e7eeb7f3f42119c0363a79febc54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4e7eeb7f3f42119c0363a79febc54b, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.114105 697 consensus_queue.cc:260] T 5ee489f17d80406d8fd5c2909612f456 P 4b4e7eeb7f3f42119c0363a79febc54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.116596 697 ts_tablet_manager.cc:1398] T 5ee489f17d80406d8fd5c2909612f456 P 4b4e7eeb7f3f42119c0363a79febc54b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:20.116782 614 ts_tablet_manager.cc:902] T c5a30e1ebd7747eb911fba8437e42077 P 4b4e7eeb7f3f42119c0363a79febc54b: Initiating tablet copy from peer f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
F20240502 07:59:20.116948 614 trace_metrics.cc:68] Check failed: g_intern_map->size() < 100 (100 vs. 100) Too many interned strings: (MaintenanceMgr.queue_time_us, MaintenanceMgr.queue_time_us) (MaintenanceMgr.run_cpu_time_us, MaintenanceMgr.run_cpu_time_us) (MaintenanceMgr.run_wall_time_us, MaintenanceMgr.run_wall_time_us) (apply.queue_time_us, apply.queue_time_us) (apply.run_cpu_time_us, apply.run_cpu_time_us) (apply.run_wall_time_us, apply.run_wall_time_us) (client-negotiator.queue_time_us, client-negotiator.queue_time_us) (client-negotiator.run_cpu_time_us, client-negotiator.run_cpu_time_us) (client-negotiator.run_wall_time_us, client-negotiator.run_wall_time_us) (compiler_manager_pool.queue_time_us, compiler_manager_pool.queue_time_us) (compiler_manager_pool.run_cpu_time_us, compiler_manager_pool.run_cpu_time_us) (compiler_manager_pool.run_wall_time_us, compiler_manager_pool.run_wall_time_us) (dirs.queue_time_us, dirs.queue_time_us) (dirs.run_cpu_time_us, dirs.run_cpu_time_us) (dirs.run_wall_time_us, dirs.run_wall_time_us) (dns-resolver.queue_time_us, dns-resolver.queue_time_us) (dns-resolver.run_cpu_time_us, dns-resolver.run_cpu_time_us) (dns-resolver.run_wall_time_us, dns-resolver.run_wall_time_us) (log-alloc.queue_time_us, log-alloc.queue_time_us) (log-alloc.run_cpu_time_us, log-alloc.run_cpu_time_us) (log-alloc.run_wall_time_us, log-alloc.run_wall_time_us) (prepare.queue_time_us, prepare.queue_time_us) (prepare.run_cpu_time_us, prepare.run_cpu_time_us) (prepare.run_wall_time_us, prepare.run_wall_time_us) (raft.queue_time_us, raft.queue_time_us) (raft.run_cpu_time_us, raft.run_cpu_time_us) (raft.run_wall_time_us, raft.run_wall_time_us) (repair_pool.queue_time_us, repair_pool.queue_time_us) (repair_pool.run_cpu_time_us, repair_pool.run_cpu_time_us) (repair_pool.run_wall_time_us, repair_pool.run_wall_time_us) (server-negotiator.queue_time_us, server-negotiator.queue_time_us) (server-negotiator.run_cpu_time_us, server-negotiator.run_cpu_time_us) (server-negotiator.run_wall_time_us, server-negotiator.run_wall_time_us) (tablet-copy.queue_time_us, tablet-copy.queue_time_us) (tablet-copy.run_cpu_time_us, tablet-copy.run_cpu_time_us) (tablet-copy.run_wall_time_us, tablet-copy.run_wall_time_us) (tablet-delete.queue_time_us, tablet-delete.queue_time_us) (tablet-delete.run_cpu_time_us, tablet-delete.run_cpu_time_us) (tablet-delete.run_wall_time_us, tablet-delete.run_wall_time_us) (tablet-download-pool-257f0f4dca624c859d3b06cfd4246536.queue_time_us, tablet-download-pool-257f0f4dca624c859d3b06cfd4246536.queue_time_us) (tablet-download-pool-257f0f4dca624c859d3b06cfd4246536.run_cpu_time_us, tablet-download-pool-257f0f4dca624c859d3b06cfd4246536.run_cpu_time_us) (tablet-download-pool-257f0f4dca624c859d3b06cfd4246536.run_wall_time_us, tablet-download-pool-257f0f4dca624c859d3b06cfd4246536.run_wall_time_us) (tablet-download-pool-423d6be23b044130b0c916f995dffced.queue_time_us, tablet-download-pool-423d6be23b044130b0c916f995dffced.queue_time_us) (tablet-download-pool-423d6be23b044130b0c916f995dffced.run_cpu_time_us, tablet-download-pool-423d6be23b044130b0c916f995dffced.run_cpu_time_us) (tablet-download-pool-423d6be23b044130b0c916f995dffced.run_wall_time_us, tablet-download-pool-423d6be23b044130b0c916f995dffced.run_wall_time_us) (tablet-download-pool-4840fa82fcb241f391c4890fb735fe9a.queue_time_us, tablet-download-pool-4840fa82fcb241f391c4890fb735fe9a.queue_time_us) (tablet-download-pool-4840fa82fcb241f391c4890fb735fe9a.run_cpu_time_us, tablet-download-pool-4840fa82fcb241f391c4890fb735fe9a.run_cpu_time_us) (tablet-download-pool-4840fa82fcb241f391c4890fb735fe9a.run_wall_time_us, tablet-download-pool-4840fa82fcb241f391c4890fb735fe9a.run_wall_time_us) (tablet-download-pool-4dc697950c8140878b50550fe0ff458d.queue_time_us, tablet-download-pool-4dc697950c8140878b50550fe0ff458d.queue_time_us) (tablet-download-pool-4dc697950c8140878b50550fe0ff458d.run_cpu_time_us, tablet-download-pool-4dc697950c8140878b50550fe0ff458d.run_cpu_time_us) (tablet-download-pool-4dc697950c8140878b50550fe0ff458d.run_wall_time_us, tablet-download-pool-4dc697950c8140878b50550fe0ff458d.run_wall_time_us) (tablet-download-pool-648a25ebebe44061bca2844ec5b048c5.queue_time_us, tablet-download-pool-648a25ebebe44061bca2844ec5b048c5.queue_time_us) (tablet-download-pool-648a25ebebe44061bca2844ec5b048c5.run_cpu_time_us, tablet-download-pool-648a25ebebe44061bca2844ec5b048c5.run_cpu_time_us) (tablet-download-pool-648a25ebebe44061bca2844ec5b048c5.run_wall_time_us, tablet-download-pool-648a25ebebe44061bca2844ec5b048c5.run_wall_time_us) (tablet-download-pool-8477b961ce384edea078851122e196e9.queue_time_us, tablet-download-pool-8477b961ce384edea078851122e196e9.queue_time_us) (tablet-download-pool-8477b961ce384edea078851122e196e9.run_cpu_time_us, tablet-download-pool-8477b961ce384edea078851122e196e9.run_cpu_time_us) (tablet-download-pool-8477b961ce384edea078851122e196e9.run_wall_time_us, tablet-download-pool-8477b961ce384edea078851122e196e9.run_wall_time_us) (tablet-download-pool-9f44c905bfaa4db093a1b39b6c21e03c.queue_time_us, tablet-download-pool-9f44c905bfaa4db093a1b39b6c21e03c.queue_time_us) (tablet-download-pool-9f44c905bfaa4db093a1b39b6c21e03c.run_cpu_time_us, tablet-download-pool-9f44c905bfaa4db093a1b39b6c21e03c.run_cpu_time_us) (tablet-download-pool-9f44c905bfaa4db093a1b39b6c21e03c.run_wall_time_us, tablet-download-pool-9f44c905bfaa4db093a1b39b6c21e03c.run_wall_time_us) (tablet-download-pool-a0430566f5ae4ffabcf15bb8b8af6bb5.queue_time_us, tablet-download-pool-a0430566f5ae4ffabcf15bb8b8af6bb5.queue_time_us) (tablet-download-pool-a0430566f5ae4ffabcf15bb8b8af6bb5.run_cpu_time_us, tablet-download-pool-a0430566f5ae4ffabcf15bb8b8af6bb5.run_cpu_time_us) (tablet-download-pool-a0430566f5ae4ffabcf15bb8b8af6bb5.run_wall_time_us, tablet-download-pool-a0430566f5ae4ffabcf15bb8b8af6bb5.run_wall_time_us) (tablet-download-pool-b270d89546214f4d9d15193f4f8e04d5.queue_time_us, tablet-download-pool-b270d89546214f4d9d15193f4f8e04d5.queue_time_us) (tablet-download-pool-b270d89546214f4d9d15193f4f8e04d5.run_cpu_time_us, tablet-download-pool-b270d89546214f4d9d15193f4f8e04d5.run_cpu_time_us) (tablet-download-pool-b270d89546214f4d9d15193f4f8e04d5.run_wall_time_us, tablet-download-pool-b270d89546214f4d9d15193f4f8e04d5.run_wall_time_us) (tablet-download-pool-c2422c666baa48078030c43d1c0938f9.queue_time_us, tablet-download-pool-c2422c666baa48078030c43d1c0938f9.queue_time_us) (tablet-download-pool-c2422c666baa48078030c43d1c0938f9.run_cpu_time_us, tablet-download-pool-c2422c666baa48078030c43d1c0938f9.run_cpu_time_us) (tablet-download-pool-c2422c666baa48078030c43d1c0938f9.run_wall_time_us, tablet-download-pool-c2422c666baa48078030c43d1c0938f9.run_wall_time_us) (tablet-download-pool-c5a30e1ebd7747eb911fba8437e42077.queue_time_us, tablet-download-pool-c5a30e1ebd7747eb911fba8437e42077.queue_time_us) (tablet-download-pool-d61cb2b6dce1409bb30d1c68c970dc22.queue_time_us, tablet-download-pool-d61cb2b6dce1409bb30d1c68c970dc22.queue_time_us) (tablet-download-pool-d61cb2b6dce1409bb30d1c68c970dc22.run_cpu_time_us, tablet-download-pool-d61cb2b6dce1409bb30d1c68c970dc22.run_cpu_time_us) (tablet-download-pool-d61cb2b6dce1409bb30d1c68c970dc22.run_wall_time_us, tablet-download-pool-d61cb2b6dce1409bb30d1c68c970dc22.run_wall_time_us) (tablet-download-pool-d63f00a2d007442781d58fb938bed480.queue_time_us, tablet-download-pool-d63f00a2d007442781d58fb938bed480.queue_time_us) (tablet-download-pool-d63f00a2d007442781d58fb938bed480.run_cpu_time_us, tablet-download-pool-d63f00a2d007442781d58fb938bed480.run_cpu_time_us) (tablet-download-pool-d63f00a2d007442781d58fb938bed480.run_wall_time_us, tablet-download-pool-d63f00a2d007442781d58fb938bed480.run_wall_time_us) (tablet-download-pool-d6dfeeb6b16a4f92bc756bf8933c7fb1.queue_time_us, tablet-download-pool-d6dfeeb6b16a4f92bc756bf8933c7fb1.queue_time_us) (tablet-download-pool-d6dfeeb6b16a4f92bc756bf8933c7fb1.run_cpu_time_us, tablet-download-pool-d6dfeeb6b16a4f92bc756bf8933c7fb1.run_cpu_time_us) (tablet-download-pool-d6dfeeb6b16a4f92bc756bf8933c7fb1.run_wall_time_us, tablet-download-pool-d6dfeeb6b16a4f92bc756bf8933c7fb1.run_wall_time_us) (tablet-download-pool-f5f0690b92c140c49fe10ff46e0136df.queue_time_us, tablet-download-pool-f5f0690b92c140c49fe10ff46e0136df.queue_time_us) (tablet-download-pool-f5f0690b92c140c49fe10ff46e0136df.run_cpu_time_us, tablet-download-pool-f5f0690b92c140c49fe10ff46e0136df.run_cpu_time_us) (tablet-download-pool-f5f0690b92c140c49fe10ff46e0136df.run_wall_time_us, tablet-download-pool-f5f0690b92c140c49fe10ff46e0136df.run_wall_time_us) (tablet-open.queue_time_us, tablet-open.queue_time_us) (tablet-open.run_cpu_time_us, tablet-open.run_cpu_time_us) (tablet-open.run_wall_time_us, tablet-open.run_wall_time_us) (txn-commit.queue_time_us, txn-commit.queue_time_us) (txn-commit.run_cpu_time_us, txn-commit.run_cpu_time_us) (txn-commit.run_wall_time_us, txn-commit.run_wall_time_us) (txn-participant-registration.queue_time_us, txn-participant-registration.queue_time_us) (txn-participant-registration.run_cpu_time_us, txn-participant-registration.run_cpu_time_us) (txn-participant-registration.run_wall_time_us, txn-participant-registration.run_wall_time_us) (txn-status-manager.queue_time_us, txn-status-manager.queue_time_us) (txn-status-manager.run_cpu_time_us, txn-status-manager.run_cpu_time_us) (txn-status-manager.run_wall_time_us, txn-status-manager.run_wall_time_us) (txn-status-tablet-reload.queue_time_us, txn-status-tablet-reload.queue_time_us) (txn-status-tablet-reload.run_cpu_time_us, txn-status-tablet-reload.run_cpu_time_us) (txn-status-tablet-reload.run_wall_time_us, txn-status-tablet-reload.run_wall_time_us) (wal-append.queue_time_us, wal-append.queue_time_us) (wal-append.run_cpu_time_us, wal-append.run_cpu_time_us) (wal-append.run_wall_time_us, wal-append.run_wall_time_us)
*** Check failure stack trace: ***
@ 0x7f94bd5f0dcd google::LogMessage::Fail() at ??:0
@ 0x7f94bd5f4b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f94bd5f07cc google::LogMessage::Flush() at ??:0
@ 0x7f94bd5f1f59 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f94bea5a044 kudu::TraceMetrics::InternName() at ??:0
@ 0x7f94bea3ef98 kudu::ThreadPool::ThreadPool() at ??:0
@ 0x7f94bea3cefc kudu::ThreadPoolBuilder::Build() at ??:0
@ 0x7f94c15ef77a kudu::tserver::TabletCopyClient::TabletCopyClient() at ??:0
@ 0x7f94c15f8a45 kudu::tserver::RemoteTabletCopyClient::RemoteTabletCopyClient() at ??:0
@ 0x7f94c167976f kudu::tserver::TSTabletManager::RunTabletCopy() at ??:0
@ 0x7f94c168d4f6 kudu::tserver::TabletCopyRunnable::Run() at ??:0
@ 0x7f94c1678226 _ZZN4kudu7tserver15TSTabletManager15StartTabletCopyEPKNS_9consensus24StartTabletCopyRequestPBESt8functionIFvRKNS_6StatusENS0_24TabletServerErrorPB_CodeEEEENKUlvE_clEv at ??:0
@ 0x7f94c1687c0a _ZNSt17_Function_handlerIFvvEZN4kudu7tserver15TSTabletManager15StartTabletCopyEPKNS1_9consensus24StartTabletCopyRequestPBESt8functionIFvRKNS1_6StatusENS2_24TabletServerErrorPB_CodeEEEEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x55e9b4d2475c std::function<>::operator()() at ??:0
@ 0x7f94bea41e06 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f94bea426ef _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f94bea4403b _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x55e9b4d2475c std::function<>::operator()() at ??:0
@ 0x7f94bea31807 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f94bf26f6db start_thread at ??:0
@ 0x7f94bc7a771f clone at ??:0
I20240502 07:59:20.203953 731 ts_tablet_manager.cc:902] T 5242aa2d2d9c477994026edb0d42051c P e51b10af49484aaa8f0267aebc1bf27f: Initiating tablet copy from peer f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:20.204596 731 tablet_copy_client.cc:308] T 5242aa2d2d9c477994026edb0d42051c P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.132:41673
I20240502 07:59:20.204993 22785 tablet_copy_service.cc:140] P f00f3a1876444ad2bf9c919068ab8014: Received BeginTabletCopySession request for tablet 5242aa2d2d9c477994026edb0d42051c from peer e51b10af49484aaa8f0267aebc1bf27f ({username='slave'} at 127.15.129.131:47637)
I20240502 07:59:20.205112 22785 tablet_copy_service.cc:161] P f00f3a1876444ad2bf9c919068ab8014: Beginning new tablet copy session on tablet 5242aa2d2d9c477994026edb0d42051c from peer e51b10af49484aaa8f0267aebc1bf27f at {username='slave'} at 127.15.129.131:47637: session id = e51b10af49484aaa8f0267aebc1bf27f-5242aa2d2d9c477994026edb0d42051c
I20240502 07:59:20.206058 22785 tablet_copy_source_session.cc:205] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:20.206851 731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5242aa2d2d9c477994026edb0d42051c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.209167 731 tablet_copy_client.cc:783] T 5242aa2d2d9c477994026edb0d42051c P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:20.209313 731 tablet_copy_client.cc:647] T 5242aa2d2d9c477994026edb0d42051c P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:20.214648 731 tablet_copy_client.cc:523] T 5242aa2d2d9c477994026edb0d42051c P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:20.215900 731 tablet_bootstrap.cc:492] T 5242aa2d2d9c477994026edb0d42051c P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:20.219836 731 tablet_bootstrap.cc:492] T 5242aa2d2d9c477994026edb0d42051c P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:20.219980 22764 raft_consensus.cc:1203] T 461098a3f8e64fe89a8603f78dc36dd2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:20.220198 731 tablet_bootstrap.cc:492] T 5242aa2d2d9c477994026edb0d42051c P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap complete.
I20240502 07:59:20.220391 731 ts_tablet_manager.cc:1367] T 5242aa2d2d9c477994026edb0d42051c P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20240502 07:59:20.220839 731 raft_consensus.cc:348] T 5242aa2d2d9c477994026edb0d42051c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:20.220949 731 raft_consensus.cc:729] T 5242aa2d2d9c477994026edb0d42051c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.221102 731 consensus_queue.cc:260] T 5242aa2d2d9c477994026edb0d42051c P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:20.221467 731 ts_tablet_manager.cc:1398] T 5242aa2d2d9c477994026edb0d42051c P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:20.221859 22785 tablet_copy_service.cc:342] P f00f3a1876444ad2bf9c919068ab8014: Request end of tablet copy session e51b10af49484aaa8f0267aebc1bf27f-5242aa2d2d9c477994026edb0d42051c received from {username='slave'} at 127.15.129.131:47637
I20240502 07:59:20.221930 22785 tablet_copy_service.cc:433] P f00f3a1876444ad2bf9c919068ab8014: ending tablet copy session e51b10af49484aaa8f0267aebc1bf27f-5242aa2d2d9c477994026edb0d42051c on tablet 5242aa2d2d9c477994026edb0d42051c with peer e51b10af49484aaa8f0267aebc1bf27f
I20240502 07:59:20.231437 22501 raft_consensus.cc:1203] T fe087f9b25c84749b7cfba34800da679 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:20.233239 32718 raft_consensus.cc:482] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:20.233373 32718 raft_consensus.cc:504] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.233842 32718 leader_election.cc:288] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:20.234344 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16764c31484a43a4a0b705ce536bde0a" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:20.234488 22632 raft_consensus.cc:2454] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:20.234705 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16764c31484a43a4a0b705ce536bde0a" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:20.234784 22701 leader_election.cc:302] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:20.234833 24952 raft_consensus.cc:2454] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:20.235082 31905 raft_consensus.cc:2781] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:20.235136 31905 raft_consensus.cc:482] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:20.235173 31905 raft_consensus.cc:3037] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:20.236271 31905 raft_consensus.cc:504] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.236604 31905 leader_election.cc:288] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:20.238312 22632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16764c31484a43a4a0b705ce536bde0a" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:20.238417 22632 raft_consensus.cc:3037] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:20.239503 22632 raft_consensus.cc:2454] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:20.239809 22701 leader_election.cc:302] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:20.240157 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16764c31484a43a4a0b705ce536bde0a" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:20.240255 32718 raft_consensus.cc:2781] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:20.240275 24952 raft_consensus.cc:3037] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:20.240341 32718 raft_consensus.cc:686] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:59:20.240485 32718 consensus_queue.cc:237] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.241298 24952 raft_consensus.cc:2454] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:20.243270 22225 catalog_manager.cc:5484] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:20.250968 731 ts_tablet_manager.cc:902] T a7da69fb73f94ce0b8c24599c8ae4d15 P e51b10af49484aaa8f0267aebc1bf27f: Initiating tablet copy from peer 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:20.251420 731 tablet_copy_client.cc:308] T a7da69fb73f94ce0b8c24599c8ae4d15 P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.136:36591
I20240502 07:59:20.251928 25235 tablet_copy_service.cc:140] P 43930dee0cd5498fa8e357a34dd71432: Received BeginTabletCopySession request for tablet a7da69fb73f94ce0b8c24599c8ae4d15 from peer e51b10af49484aaa8f0267aebc1bf27f ({username='slave'} at 127.15.129.131:60465)
I20240502 07:59:20.252027 25235 tablet_copy_service.cc:161] P 43930dee0cd5498fa8e357a34dd71432: Beginning new tablet copy session on tablet a7da69fb73f94ce0b8c24599c8ae4d15 from peer e51b10af49484aaa8f0267aebc1bf27f at {username='slave'} at 127.15.129.131:60465: session id = e51b10af49484aaa8f0267aebc1bf27f-a7da69fb73f94ce0b8c24599c8ae4d15
I20240502 07:59:20.253168 25235 tablet_copy_source_session.cc:205] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:20.254349 731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7da69fb73f94ce0b8c24599c8ae4d15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.256191 731 tablet_copy_client.cc:783] T a7da69fb73f94ce0b8c24599c8ae4d15 P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:20.256327 731 tablet_copy_client.cc:647] T a7da69fb73f94ce0b8c24599c8ae4d15 P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:20.257434 731 tablet_copy_client.cc:523] T a7da69fb73f94ce0b8c24599c8ae4d15 P e51b10af49484aaa8f0267aebc1bf27f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:20.258747 731 tablet_bootstrap.cc:492] T a7da69fb73f94ce0b8c24599c8ae4d15 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:20.263082 731 tablet_bootstrap.cc:492] T a7da69fb73f94ce0b8c24599c8ae4d15 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:20.263322 731 tablet_bootstrap.cc:492] T a7da69fb73f94ce0b8c24599c8ae4d15 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap complete.
I20240502 07:59:20.263495 731 ts_tablet_manager.cc:1367] T a7da69fb73f94ce0b8c24599c8ae4d15 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20240502 07:59:20.263942 731 raft_consensus.cc:348] T a7da69fb73f94ce0b8c24599c8ae4d15 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:20.264060 731 raft_consensus.cc:729] T a7da69fb73f94ce0b8c24599c8ae4d15 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.264209 731 consensus_queue.cc:260] T a7da69fb73f94ce0b8c24599c8ae4d15 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:20.264473 731 ts_tablet_manager.cc:1398] T a7da69fb73f94ce0b8c24599c8ae4d15 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240502 07:59:20.264952 25235 tablet_copy_service.cc:342] P 43930dee0cd5498fa8e357a34dd71432: Request end of tablet copy session e51b10af49484aaa8f0267aebc1bf27f-a7da69fb73f94ce0b8c24599c8ae4d15 received from {username='slave'} at 127.15.129.131:60465
I20240502 07:59:20.265030 25235 tablet_copy_service.cc:433] P 43930dee0cd5498fa8e357a34dd71432: ending tablet copy session e51b10af49484aaa8f0267aebc1bf27f-a7da69fb73f94ce0b8c24599c8ae4d15 on tablet a7da69fb73f94ce0b8c24599c8ae4d15 with peer e51b10af49484aaa8f0267aebc1bf27f
I20240502 07:59:20.270083 24930 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:20.282686 25193 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:20.287081 761 ts_tablet_manager.cc:902] T 2241a741bf85433b9fea512e9c530416 P 227eec4b5f014d349661f6e0b39b9055: Initiating tablet copy from peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:20.287611 761 tablet_copy_client.cc:308] T 2241a741bf85433b9fea512e9c530416 P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.131:43703
I20240502 07:59:20.288331 22653 tablet_copy_service.cc:140] P e51b10af49484aaa8f0267aebc1bf27f: Received BeginTabletCopySession request for tablet 2241a741bf85433b9fea512e9c530416 from peer 227eec4b5f014d349661f6e0b39b9055 ({username='slave'} at 127.15.129.135:39233)
I20240502 07:59:20.288431 22653 tablet_copy_service.cc:161] P e51b10af49484aaa8f0267aebc1bf27f: Beginning new tablet copy session on tablet 2241a741bf85433b9fea512e9c530416 from peer 227eec4b5f014d349661f6e0b39b9055 at {username='slave'} at 127.15.129.135:39233: session id = 227eec4b5f014d349661f6e0b39b9055-2241a741bf85433b9fea512e9c530416
I20240502 07:59:20.289501 22653 tablet_copy_source_session.cc:205] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:20.290793 761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2241a741bf85433b9fea512e9c530416. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.295400 761 tablet_copy_client.cc:783] T 2241a741bf85433b9fea512e9c530416 P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:20.295558 761 tablet_copy_client.cc:647] T 2241a741bf85433b9fea512e9c530416 P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:20.296429 25063 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:20.297749 22481 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:20.299441 22613 tablet_service.cc:1429] Tablet server has 22 leaders and 0 scanners
I20240502 07:59:20.323282 766 ts_tablet_manager.cc:902] T 2ea24049cec547b0a6cb85be6033eb3d P 1aecef81733c47b3b468504934357231: Initiating tablet copy from peer 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:20.323843 766 tablet_copy_client.cc:308] T 2ea24049cec547b0a6cb85be6033eb3d P 1aecef81733c47b3b468504934357231: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.136:36591
I20240502 07:59:20.324259 25235 tablet_copy_service.cc:140] P 43930dee0cd5498fa8e357a34dd71432: Received BeginTabletCopySession request for tablet 2ea24049cec547b0a6cb85be6033eb3d from peer 1aecef81733c47b3b468504934357231 ({username='slave'} at 127.15.129.134:52175)
I20240502 07:59:20.324376 25235 tablet_copy_service.cc:161] P 43930dee0cd5498fa8e357a34dd71432: Beginning new tablet copy session on tablet 2ea24049cec547b0a6cb85be6033eb3d from peer 1aecef81733c47b3b468504934357231 at {username='slave'} at 127.15.129.134:52175: session id = 1aecef81733c47b3b468504934357231-2ea24049cec547b0a6cb85be6033eb3d
I20240502 07:59:20.324503 22351 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:20.325464 25235 tablet_copy_source_session.cc:205] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:20.326824 766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ea24049cec547b0a6cb85be6033eb3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.327606 22743 tablet_service.cc:1429] Tablet server has 11 leaders and 0 scanners
I20240502 07:59:20.330199 761 tablet_copy_client.cc:523] T 2241a741bf85433b9fea512e9c530416 P 227eec4b5f014d349661f6e0b39b9055: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:20.331544 761 tablet_bootstrap.cc:492] T 2241a741bf85433b9fea512e9c530416 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:20.337249 766 tablet_copy_client.cc:783] T 2ea24049cec547b0a6cb85be6033eb3d P 1aecef81733c47b3b468504934357231: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:20.337445 766 tablet_copy_client.cc:647] T 2ea24049cec547b0a6cb85be6033eb3d P 1aecef81733c47b3b468504934357231: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:20.353770 766 tablet_copy_client.cc:523] T 2ea24049cec547b0a6cb85be6033eb3d P 1aecef81733c47b3b468504934357231: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:20.355096 766 tablet_bootstrap.cc:492] T 2ea24049cec547b0a6cb85be6033eb3d P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:20.356391 31905 raft_consensus.cc:482] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:20.356487 31905 raft_consensus.cc:504] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.356886 31905 leader_election.cc:288] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:20.357266 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "497e7c78a2274cc1a8406cdab125e0d3" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:20.357270 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "497e7c78a2274cc1a8406cdab125e0d3" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:20.357407 25213 raft_consensus.cc:2454] T 497e7c78a2274cc1a8406cdab125e0d3 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:20.357434 22370 raft_consensus.cc:2454] T 497e7c78a2274cc1a8406cdab125e0d3 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:20.357736 22701 leader_election.cc:302] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:20.357894 31905 raft_consensus.cc:2781] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:20.357954 31905 raft_consensus.cc:482] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:20.357990 31905 raft_consensus.cc:3037] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:20.359037 31905 raft_consensus.cc:504] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.359371 31905 leader_election.cc:288] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:20.360044 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "497e7c78a2274cc1a8406cdab125e0d3" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:20.360173 22370 raft_consensus.cc:3037] T 497e7c78a2274cc1a8406cdab125e0d3 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:20.361289 22370 raft_consensus.cc:2454] T 497e7c78a2274cc1a8406cdab125e0d3 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:20.363883 22701 leader_election.cc:302] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:20.364059 31905 raft_consensus.cc:2781] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:20.364156 31905 raft_consensus.cc:686] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:59:20.364266 31905 consensus_queue.cc:237] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.365890 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "497e7c78a2274cc1a8406cdab125e0d3" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:20.365995 25213 raft_consensus.cc:3037] T 497e7c78a2274cc1a8406cdab125e0d3 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:20.366923 25213 raft_consensus.cc:2454] T 497e7c78a2274cc1a8406cdab125e0d3 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:20.367223 22225 catalog_manager.cc:5484] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:20.376883 774 ts_tablet_manager.cc:902] T 038a695352ca4da58afaeb63a7785866 P b533a7d8d97943609c6e28e3e36b78a4: Initiating tablet copy from peer f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:20.377509 774 tablet_copy_client.cc:308] T 038a695352ca4da58afaeb63a7785866 P b533a7d8d97943609c6e28e3e36b78a4: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.132:41673
I20240502 07:59:20.377768 25213 raft_consensus.cc:1203] T 45925e9713cb4b748efde06f77cf3702 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240502 07:59:20.379056 22785 tablet_copy_service.cc:140] P f00f3a1876444ad2bf9c919068ab8014: Received BeginTabletCopySession request for tablet 038a695352ca4da58afaeb63a7785866 from peer b533a7d8d97943609c6e28e3e36b78a4 ({username='slave'} at 127.15.129.129:39449)
I20240502 07:59:20.379168 22785 tablet_copy_service.cc:161] P f00f3a1876444ad2bf9c919068ab8014: Beginning new tablet copy session on tablet 038a695352ca4da58afaeb63a7785866 from peer b533a7d8d97943609c6e28e3e36b78a4 at {username='slave'} at 127.15.129.129:39449: session id = b533a7d8d97943609c6e28e3e36b78a4-038a695352ca4da58afaeb63a7785866
I20240502 07:59:20.380198 22785 tablet_copy_source_session.cc:205] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:20.383626 774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 038a695352ca4da58afaeb63a7785866. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.386023 774 tablet_copy_client.cc:783] T 038a695352ca4da58afaeb63a7785866 P b533a7d8d97943609c6e28e3e36b78a4: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:20.386188 774 tablet_copy_client.cc:647] T 038a695352ca4da58afaeb63a7785866 P b533a7d8d97943609c6e28e3e36b78a4: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:20.386559 22225 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:20.387300 774 tablet_copy_client.cc:523] T 038a695352ca4da58afaeb63a7785866 P b533a7d8d97943609c6e28e3e36b78a4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:20.388501 774 tablet_bootstrap.cc:492] T 038a695352ca4da58afaeb63a7785866 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:20.399344 24952 raft_consensus.cc:1261] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:20.399814 22632 raft_consensus.cc:1261] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:20.399982 774 tablet_bootstrap.cc:492] T 038a695352ca4da58afaeb63a7785866 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:20.400254 774 tablet_bootstrap.cc:492] T 038a695352ca4da58afaeb63a7785866 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap complete.
I20240502 07:59:20.400446 774 ts_tablet_manager.cc:1367] T 038a695352ca4da58afaeb63a7785866 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.001s
I20240502 07:59:20.400939 774 raft_consensus.cc:348] T 038a695352ca4da58afaeb63a7785866 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:20.401054 774 raft_consensus.cc:729] T 038a695352ca4da58afaeb63a7785866 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.401036 32718 consensus_queue.cc:1035] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:20.401176 774 consensus_queue.cc:260] T 038a695352ca4da58afaeb63a7785866 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:20.401386 31905 consensus_queue.cc:1035] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:20.401511 774 ts_tablet_manager.cc:1398] T 038a695352ca4da58afaeb63a7785866 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 07:59:20.402181 22785 tablet_copy_service.cc:342] P f00f3a1876444ad2bf9c919068ab8014: Request end of tablet copy session b533a7d8d97943609c6e28e3e36b78a4-038a695352ca4da58afaeb63a7785866 received from {username='slave'} at 127.15.129.129:39449
I20240502 07:59:20.402247 22785 tablet_copy_service.cc:433] P f00f3a1876444ad2bf9c919068ab8014: ending tablet copy session b533a7d8d97943609c6e28e3e36b78a4-038a695352ca4da58afaeb63a7785866 on tablet 038a695352ca4da58afaeb63a7785866 with peer b533a7d8d97943609c6e28e3e36b78a4
I20240502 07:59:20.406443 22222 catalog_manager.cc:5576] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703) does not have the latest schema for tablet 16764c31484a43a4a0b705ce536bde0a (table rebalancer_dynamic_table [id=a30a987f1fcf4907a3a5ed5f97997f7c]). Expected version 1 got 0
I20240502 07:59:20.408776 766 tablet_bootstrap.cc:492] T 2ea24049cec547b0a6cb85be6033eb3d P 1aecef81733c47b3b468504934357231: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:20.409080 766 tablet_bootstrap.cc:492] T 2ea24049cec547b0a6cb85be6033eb3d P 1aecef81733c47b3b468504934357231: Bootstrap complete.
I20240502 07:59:20.409284 766 ts_tablet_manager.cc:1367] T 2ea24049cec547b0a6cb85be6033eb3d P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.054s user 0.002s sys 0.002s
I20240502 07:59:20.409782 766 raft_consensus.cc:348] T 2ea24049cec547b0a6cb85be6033eb3d P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:20.410140 766 raft_consensus.cc:729] T 2ea24049cec547b0a6cb85be6033eb3d P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.410298 766 consensus_queue.cc:260] T 2ea24049cec547b0a6cb85be6033eb3d P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:20.410609 766 ts_tablet_manager.cc:1398] T 2ea24049cec547b0a6cb85be6033eb3d P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:20.410917 22222 catalog_manager.cc:5576] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995) does not have the latest schema for tablet 16764c31484a43a4a0b705ce536bde0a (table rebalancer_dynamic_table [id=a30a987f1fcf4907a3a5ed5f97997f7c]). Expected version 1 got 0
I20240502 07:59:20.412427 25235 tablet_copy_service.cc:342] P 43930dee0cd5498fa8e357a34dd71432: Request end of tablet copy session 1aecef81733c47b3b468504934357231-2ea24049cec547b0a6cb85be6033eb3d received from {username='slave'} at 127.15.129.134:52175
I20240502 07:59:20.412506 25235 tablet_copy_service.cc:433] P 43930dee0cd5498fa8e357a34dd71432: ending tablet copy session 1aecef81733c47b3b468504934357231-2ea24049cec547b0a6cb85be6033eb3d on tablet 2ea24049cec547b0a6cb85be6033eb3d with peer 1aecef81733c47b3b468504934357231
I20240502 07:59:20.414736 785 tablet.cc:1722] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:20.414851 25213 raft_consensus.cc:1203] T 895f9a31785d4558acd3b64904a645ca P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240502 07:59:20.415055 785 tablet.cc:1620] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:59:20.416209 761 tablet_bootstrap.cc:492] T 2241a741bf85433b9fea512e9c530416 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=676 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:20.416498 761 tablet_bootstrap.cc:492] T 2241a741bf85433b9fea512e9c530416 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap complete.
I20240502 07:59:20.416646 761 ts_tablet_manager.cc:1367] T 2241a741bf85433b9fea512e9c530416 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.085s user 0.026s sys 0.000s
I20240502 07:59:20.417022 761 raft_consensus.cc:348] T 2241a741bf85433b9fea512e9c530416 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:20.417131 761 raft_consensus.cc:729] T 2241a741bf85433b9fea512e9c530416 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.422094 22225 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=a30a987f1fcf4907a3a5ed5f97997f7c] alter complete (version 1)
I20240502 07:59:20.426188 761 consensus_queue.cc:260] T 2241a741bf85433b9fea512e9c530416 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:20.427728 761 ts_tablet_manager.cc:1398] T 2241a741bf85433b9fea512e9c530416 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20240502 07:59:20.427739 22227 catalog_manager.cc:5576] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703) does not have the latest schema for tablet 16764c31484a43a4a0b705ce536bde0a (table rebalancer_dynamic_table [id=a30a987f1fcf4907a3a5ed5f97997f7c]). Expected version 1 got 0
I20240502 07:59:20.430083 365 raft_consensus.cc:482] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:20.430187 365 raft_consensus.cc:504] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:20.430361 786 tablet.cc:1722] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:20.430622 365 leader_election.cc:288] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:20.430693 786 tablet.cc:1620] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
I20240502 07:59:20.430995 22653 tablet_copy_service.cc:342] P e51b10af49484aaa8f0267aebc1bf27f: Request end of tablet copy session 227eec4b5f014d349661f6e0b39b9055-2241a741bf85433b9fea512e9c530416 received from {username='slave'} at 127.15.129.135:39233
I20240502 07:59:20.431067 22653 tablet_copy_service.cc:433] P e51b10af49484aaa8f0267aebc1bf27f: ending tablet copy session 227eec4b5f014d349661f6e0b39b9055-2241a741bf85433b9fea512e9c530416 on tablet 2241a741bf85433b9fea512e9c530416 with peer 227eec4b5f014d349661f6e0b39b9055
I20240502 07:59:20.431914 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a511340ea9e649c690b11afb9a35835f" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:20.432051 22370 raft_consensus.cc:2454] T a511340ea9e649c690b11afb9a35835f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:20.432339 25020 leader_election.cc:302] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:20.433938 365 raft_consensus.cc:2781] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:20.434005 365 raft_consensus.cc:482] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:20.434044 365 raft_consensus.cc:3037] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:20.435163 365 raft_consensus.cc:504] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:20.435576 365 leader_election.cc:288] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:20.436183 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a511340ea9e649c690b11afb9a35835f" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:20.436306 22370 raft_consensus.cc:3037] T a511340ea9e649c690b11afb9a35835f P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:20.442296 22370 raft_consensus.cc:2454] T a511340ea9e649c690b11afb9a35835f P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:20.442608 789 tablet.cc:1722] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:20.442730 25020 leader_election.cc:302] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:20.442955 789 tablet.cc:1620] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:20.443091 32292 raft_consensus.cc:2781] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:20.443198 32292 raft_consensus.cc:686] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:20.443327 32292 consensus_queue.cc:237] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:20.448786 22225 catalog_manager.cc:5484] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:20.451937 22501 raft_consensus.cc:1203] T f033e8bbac8445f3af64faa67c818987 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:20.457464 774 ts_tablet_manager.cc:902] T 5d342c368796426c81bede7ddc23937b P b533a7d8d97943609c6e28e3e36b78a4: Initiating tablet copy from peer 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:20.457952 774 tablet_copy_client.cc:308] T 5d342c368796426c81bede7ddc23937b P b533a7d8d97943609c6e28e3e36b78a4: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.135:42733
I20240502 07:59:20.458391 25104 tablet_copy_service.cc:140] P 227eec4b5f014d349661f6e0b39b9055: Received BeginTabletCopySession request for tablet 5d342c368796426c81bede7ddc23937b from peer b533a7d8d97943609c6e28e3e36b78a4 ({username='slave'} at 127.15.129.129:47749)
I20240502 07:59:20.458485 25104 tablet_copy_service.cc:161] P 227eec4b5f014d349661f6e0b39b9055: Beginning new tablet copy session on tablet 5d342c368796426c81bede7ddc23937b from peer b533a7d8d97943609c6e28e3e36b78a4 at {username='slave'} at 127.15.129.129:47749: session id = b533a7d8d97943609c6e28e3e36b78a4-5d342c368796426c81bede7ddc23937b
I20240502 07:59:20.458540 31905 raft_consensus.cc:482] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:20.458624 31905 raft_consensus.cc:504] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.458999 31905 leader_election.cc:288] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:20.459496 25104 tablet_copy_source_session.cc:205] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055: Tablet Copy: opened 0 blocks and 1 log segments
I20240502 07:59:20.459723 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ee489f17d80406d8fd5c2909612f456" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:20.459853 25084 raft_consensus.cc:2454] T 5ee489f17d80406d8fd5c2909612f456 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:20.460162 22698 leader_election.cc:302] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:20.460709 774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d342c368796426c81bede7ddc23937b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.462913 774 tablet_copy_client.cc:783] T 5d342c368796426c81bede7ddc23937b P b533a7d8d97943609c6e28e3e36b78a4: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:20.463094 774 tablet_copy_client.cc:647] T 5d342c368796426c81bede7ddc23937b P b533a7d8d97943609c6e28e3e36b78a4: tablet copy: Starting download of 1 WAL segments...
I20240502 07:59:20.467371 31905 raft_consensus.cc:2781] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:20.467442 31905 raft_consensus.cc:482] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:20.467490 31905 raft_consensus.cc:3037] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:20.468627 31905 raft_consensus.cc:504] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.469024 31905 leader_election.cc:288] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:20.469498 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ee489f17d80406d8fd5c2909612f456" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:20.469610 25084 raft_consensus.cc:3037] T 5ee489f17d80406d8fd5c2909612f456 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:20.470638 25084 raft_consensus.cc:2454] T 5ee489f17d80406d8fd5c2909612f456 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:20.470966 22698 leader_election.cc:302] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:20.474040 31905 raft_consensus.cc:2781] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:20.474213 31905 raft_consensus.cc:686] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:59:20.474345 31905 consensus_queue.cc:237] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:20.476642 22222 catalog_manager.cc:5484] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:20.480427 774 tablet_copy_client.cc:523] T 5d342c368796426c81bede7ddc23937b P b533a7d8d97943609c6e28e3e36b78a4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240502 07:59:20.482498 774 tablet_bootstrap.cc:492] T 5d342c368796426c81bede7ddc23937b P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:20.487752 774 tablet_bootstrap.cc:492] T 5d342c368796426c81bede7ddc23937b P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:20.488025 774 tablet_bootstrap.cc:492] T 5d342c368796426c81bede7ddc23937b P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap complete.
I20240502 07:59:20.488204 774 ts_tablet_manager.cc:1367] T 5d342c368796426c81bede7ddc23937b P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240502 07:59:20.488798 774 raft_consensus.cc:348] T 5d342c368796426c81bede7ddc23937b P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:20.488929 774 raft_consensus.cc:729] T 5d342c368796426c81bede7ddc23937b P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.489085 774 consensus_queue.cc:260] T 5d342c368796426c81bede7ddc23937b P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:20.489408 774 ts_tablet_manager.cc:1398] T 5d342c368796426c81bede7ddc23937b P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20240502 07:59:20.490012 25104 tablet_copy_service.cc:342] P 227eec4b5f014d349661f6e0b39b9055: Request end of tablet copy session b533a7d8d97943609c6e28e3e36b78a4-5d342c368796426c81bede7ddc23937b received from {username='slave'} at 127.15.129.129:47749
I20240502 07:59:20.490092 25104 tablet_copy_service.cc:433] P 227eec4b5f014d349661f6e0b39b9055: ending tablet copy session b533a7d8d97943609c6e28e3e36b78a4-5d342c368796426c81bede7ddc23937b on tablet 5d342c368796426c81bede7ddc23937b with peer b533a7d8d97943609c6e28e3e36b78a4
I20240502 07:59:20.493885 495 ts_tablet_manager.cc:902] T caf2dd9c96b94566b216692a9370f16e P fabddb16c85944648a710ec52fc7244c: Initiating tablet copy from peer 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:20.494393 495 tablet_copy_client.cc:308] T caf2dd9c96b94566b216692a9370f16e P fabddb16c85944648a710ec52fc7244c: tablet copy: Beginning tablet copy session from remote peer at address 127.15.129.134:35995
I20240502 07:59:20.494854 24972 tablet_copy_service.cc:140] P 1aecef81733c47b3b468504934357231: Received BeginTabletCopySession request for tablet caf2dd9c96b94566b216692a9370f16e from peer fabddb16c85944648a710ec52fc7244c ({username='slave'} at 127.15.129.130:52777)
W20240502 07:59:20.505430 25150 connection.cc:535] client connection to 127.15.129.133:41661 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 07:59:20.505097 22569 connection.cc:535] client connection to 127.15.129.133:41661 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 07:59:20.517854 22569 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 07:59:20.517855 25150 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 07:59:20.505398 25019 connection.cc:535] client connection to 127.15.129.133:41661 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 07:59:20.518378 25019 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 07:59:20.518762 22569 consensus_peers.cc:462] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:20.505024 22307 connection.cc:535] client connection to 127.15.129.133:41661 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 07:59:20.518821 22569 consensus_peers.cc:485] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.518862 22569 consensus_peers.cc:485] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.518898 22569 consensus_peers.cc:485] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.518913 22307 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 07:59:20.519977 22569 consensus_peers.cc:485] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.505085 22438 connection.cc:535] client connection to 127.15.129.133:41661 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 07:59:20.520164 22438 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 07:59:20.520490 22307 consensus_peers.cc:485] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.520548 22307 consensus_peers.cc:485] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.504901 22700 connection.cc:535] client connection to 127.15.129.133:41661 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 07:59:20.520574 22307 consensus_peers.cc:485] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.520653 22700 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20240502 07:59:20.494953 24972 tablet_copy_service.cc:161] P 1aecef81733c47b3b468504934357231: Beginning new tablet copy session on tablet caf2dd9c96b94566b216692a9370f16e from peer fabddb16c85944648a710ec52fc7244c at {username='slave'} at 127.15.129.130:52777: session id = fabddb16c85944648a710ec52fc7244c-caf2dd9c96b94566b216692a9370f16e
W20240502 07:59:20.521138 22307 consensus_peers.cc:485] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:20.521842 24972 tablet_copy_source_session.cc:205] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231: Tablet Copy: opened 0 blocks and 1 log segments
W20240502 07:59:20.522097 22700 leader_election.cc:334] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:20.505717 24887 connection.cc:535] client connection to 127.15.129.133:41661 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 07:59:20.522236 22700 consensus_peers.cc:462] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:20.522307 24887 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 07:59:20.522329 22700 leader_election.cc:334] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:20.523047 22700 consensus_peers.cc:485] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:20.523738 495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caf2dd9c96b94566b216692a9370f16e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.525952 495 tablet_copy_client.cc:783] T caf2dd9c96b94566b216692a9370f16e P fabddb16c85944648a710ec52fc7244c: tablet copy: Starting download of 0 data blocks...
I20240502 07:59:20.526108 495 tablet_copy_client.cc:647] T caf2dd9c96b94566b216692a9370f16e P fabddb16c85944648a710ec52fc7244c: tablet copy: Starting download of 1 WAL segments...
W20240502 07:59:20.527537 22438 consensus_peers.cc:485] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:20.527597 495 tablet_copy_client.cc:523] T caf2dd9c96b94566b216692a9370f16e P fabddb16c85944648a710ec52fc7244c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20240502 07:59:20.527607 22438 consensus_peers.cc:485] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.527716 22438 consensus_peers.cc:485] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:20.528853 495 tablet_bootstrap.cc:492] T caf2dd9c96b94566b216692a9370f16e P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:20.531986 495 tablet_bootstrap.cc:492] T caf2dd9c96b94566b216692a9370f16e P fabddb16c85944648a710ec52fc7244c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240502 07:59:20.532229 495 tablet_bootstrap.cc:492] T caf2dd9c96b94566b216692a9370f16e P fabddb16c85944648a710ec52fc7244c: Bootstrap complete.
I20240502 07:59:20.532369 495 ts_tablet_manager.cc:1367] T caf2dd9c96b94566b216692a9370f16e P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240502 07:59:20.532796 495 raft_consensus.cc:348] T caf2dd9c96b94566b216692a9370f16e P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:20.532959 495 raft_consensus.cc:729] T caf2dd9c96b94566b216692a9370f16e P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.533130 495 consensus_queue.cc:260] T caf2dd9c96b94566b216692a9370f16e P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:20.533598 495 ts_tablet_manager.cc:1398] T caf2dd9c96b94566b216692a9370f16e P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
W20240502 07:59:20.533686 25150 consensus_peers.cc:485] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.533753 25150 consensus_peers.cc:485] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.533794 25150 consensus_peers.cc:485] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.533833 25150 consensus_peers.cc:485] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.533860 25150 consensus_peers.cc:462] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:20.534646 24972 tablet_copy_service.cc:342] P 1aecef81733c47b3b468504934357231: Request end of tablet copy session fabddb16c85944648a710ec52fc7244c-caf2dd9c96b94566b216692a9370f16e received from {username='slave'} at 127.15.129.130:52777
I20240502 07:59:20.534727 24972 tablet_copy_service.cc:433] P 1aecef81733c47b3b468504934357231: ending tablet copy session fabddb16c85944648a710ec52fc7244c-caf2dd9c96b94566b216692a9370f16e on tablet caf2dd9c96b94566b216692a9370f16e with peer fabddb16c85944648a710ec52fc7244c
W20240502 07:59:20.542145 24887 consensus_peers.cc:485] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.542260 24887 consensus_peers.cc:485] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.542308 24887 consensus_peers.cc:485] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.542343 24887 consensus_peers.cc:485] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.543210 25019 leader_election.cc:334] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:20.543748 25019 consensus_peers.cc:485] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.543812 25019 consensus_peers.cc:485] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.543848 25019 leader_election.cc:334] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:20.543972 25019 consensus_peers.cc:485] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:20.557664 22222 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:20.559016 22764 raft_consensus.cc:1203] T 26b5df5842324628b2e9bb6dc1fc36f2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:20.562697 22501 raft_consensus.cc:1203] T ff6bc97390164a37bcdaafd651c888cf P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:20.564985 785 tablet.cc:1722] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:20.565348 785 tablet.cc:1620] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:59:20.565647 789 tablet.cc:1722] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:20.565922 789 tablet.cc:1620] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:20.566727 786 tablet.cc:1722] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:20.566995 786 tablet.cc:1620] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
W20240502 07:59:20.572927 24887 consensus_peers.cc:485] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.574671 22307 consensus_peers.cc:485] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:20.575695 22227 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=a30a987f1fcf4907a3a5ed5f97997f7c] alter complete (version 2)
W20240502 07:59:20.580715 22307 consensus_peers.cc:485] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.588156 25150 consensus_peers.cc:485] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.600783 25150 consensus_peers.cc:485] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.626411 25019 consensus_peers.cc:485] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.627032 22438 consensus_peers.cc:485] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.641690 22569 consensus_peers.cc:485] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:20.679410 22230 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:20.686950 25193 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:20.688457 785 tablet.cc:1722] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:20.689891 25063 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:20.690049 789 tablet.cc:1722] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:20.694322 22230 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=a30a987f1fcf4907a3a5ed5f97997f7c] alter complete (version 3)
I20240502 07:59:20.700767 786 tablet.cc:1722] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:20.705437 24930 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:20.705699 22743 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
W20240502 07:59:20.706055 22700 consensus_peers.cc:462] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:20.710875 22481 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:20.715924 22613 tablet_service.cc:1429] Tablet server has 22 leaders and 0 scanners
I20240502 07:59:20.746618 22351 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
W20240502 07:59:20.750530 22438 consensus_peers.cc:485] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.754364 25150 consensus_peers.cc:485] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:20.762214 22569 consensus_peers.cc:462] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:20.763612 22569 consensus_peers.cc:485] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:20.779736 22370 raft_consensus.cc:1203] T 038a695352ca4da58afaeb63a7785866 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:20.786895 25084 raft_consensus.cc:1203] T 2241a741bf85433b9fea512e9c530416 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.5->[1.6-1.6] Dedup: 1.6->[]
I20240502 07:59:20.793833 22632 raft_consensus.cc:1203] T 5242aa2d2d9c477994026edb0d42051c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:20.797420 22230 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:20.797547 22230 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:20.799861 22230 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 16764c31484a43a4a0b705ce536bde0a
I20240502 07:59:20.800956 22743 tablet_service.cc:1514] Processing DeleteTablet for tablet 16764c31484a43a4a0b705ce536bde0a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:20 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:20.801227 24930 tablet_service.cc:1514] Processing DeleteTablet for tablet 16764c31484a43a4a0b705ce536bde0a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:20 UTC) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:20.801517 890 tablet_replica.cc:330] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:20.801892 890 raft_consensus.cc:2229] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:20.801918 22613 tablet_service.cc:1514] Processing DeleteTablet for tablet 16764c31484a43a4a0b705ce536bde0a with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:20 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:20.802073 890 raft_consensus.cc:2258] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:20.802189 891 tablet_replica.cc:330] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:20.802330 891 raft_consensus.cc:2229] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:20.802430 891 raft_consensus.cc:2258] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:20.802822 890 ts_tablet_manager.cc:1875] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:20.803233 889 tablet_replica.cc:330] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:20.803421 889 raft_consensus.cc:2229] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:20.803763 889 raft_consensus.cc:2258] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:20.804401 889 ts_tablet_manager.cc:1875] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:20.804462 891 ts_tablet_manager.cc:1875] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:20.805651 890 ts_tablet_manager.cc:1888] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:20.805862 890 log.cc:1196] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/16764c31484a43a4a0b705ce536bde0a
I20240502 07:59:20.806257 890 ts_tablet_manager.cc:1909] T 16764c31484a43a4a0b705ce536bde0a P 1aecef81733c47b3b468504934357231: Deleting consensus metadata
I20240502 07:59:20.806665 889 ts_tablet_manager.cc:1888] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:20.806800 889 log.cc:1196] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/16764c31484a43a4a0b705ce536bde0a
I20240502 07:59:20.807071 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet 16764c31484a43a4a0b705ce536bde0a (table rebalancer_dynamic_table_new_name [id=a30a987f1fcf4907a3a5ed5f97997f7c]) successfully deleted
I20240502 07:59:20.807204 889 ts_tablet_manager.cc:1909] T 16764c31484a43a4a0b705ce536bde0a P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:59:20.807898 891 ts_tablet_manager.cc:1888] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:20.808029 891 log.cc:1196] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/16764c31484a43a4a0b705ce536bde0a
I20240502 07:59:20.808321 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 16764c31484a43a4a0b705ce536bde0a (table rebalancer_dynamic_table_new_name [id=a30a987f1fcf4907a3a5ed5f97997f7c]) successfully deleted
I20240502 07:59:20.808959 891 ts_tablet_manager.cc:1909] T 16764c31484a43a4a0b705ce536bde0a P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:59:20.810830 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 16764c31484a43a4a0b705ce536bde0a (table rebalancer_dynamic_table_new_name [id=a30a987f1fcf4907a3a5ed5f97997f7c]) successfully deleted
I20240502 07:59:20.862800 22370 raft_consensus.cc:1203] T 5d342c368796426c81bede7ddc23937b P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:20.867820 22632 raft_consensus.cc:1203] T a7da69fb73f94ce0b8c24599c8ae4d15 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20240502 07:59:20.870056 25150 consensus_peers.cc:462] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:20.898461 32718 consensus_queue.cc:1035] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:20.901695 22230 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:20.905743 25193 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:20.906212 25063 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:20.916057 22613 tablet_service.cc:1429] Tablet server has 22 leaders and 0 scanners
I20240502 07:59:20.918318 22743 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:20.928018 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 85d8bd64c2c544c0ad14a4e89bb82d05 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=daee501832934f049a243f2024abda3d]), partition=
I20240502 07:59:20.928057 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 85d8bd64c2c544c0ad14a4e89bb82d05 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=daee501832934f049a243f2024abda3d]), partition=
I20240502 07:59:20.928241 25193 tablet_service.cc:1467] Processing CreateTablet for tablet 85d8bd64c2c544c0ad14a4e89bb82d05 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=daee501832934f049a243f2024abda3d]), partition=
I20240502 07:59:20.928282 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85d8bd64c2c544c0ad14a4e89bb82d05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.928287 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85d8bd64c2c544c0ad14a4e89bb82d05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.928449 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85d8bd64c2c544c0ad14a4e89bb82d05. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:20.930896 22481 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:20.932353 935 tablet_bootstrap.cc:492] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:20.933250 935 tablet_bootstrap.cc:654] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:20.933501 936 tablet_bootstrap.cc:492] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:20.934492 934 tablet_bootstrap.cc:492] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:20.938208 24952 raft_consensus.cc:1203] T 2ea24049cec547b0a6cb85be6033eb3d P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:20.945799 936 tablet_bootstrap.cc:654] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:20.946700 934 tablet_bootstrap.cc:654] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:20.950042 935 tablet_bootstrap.cc:492] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:20.950191 935 ts_tablet_manager.cc:1367] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 07:59:20.950701 935 raft_consensus.cc:348] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:20.950903 935 raft_consensus.cc:374] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:20.950978 935 raft_consensus.cc:729] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.951229 935 consensus_queue.cc:260] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:20.952302 22351 tablet_service.cc:1429] Tablet server has 12 leaders and 0 scanners
I20240502 07:59:20.953717 935 ts_tablet_manager.cc:1398] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
W20240502 07:59:20.959218 22700 consensus_peers.cc:485] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:20.971459 24930 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:20.974375 31905 consensus_queue.cc:1035] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:20.982322 936 tablet_bootstrap.cc:492] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:20.982450 936 ts_tablet_manager.cc:1367] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.049s user 0.002s sys 0.000s
I20240502 07:59:20.982998 936 raft_consensus.cc:348] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:20.983127 936 raft_consensus.cc:374] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:20.983176 936 raft_consensus.cc:729] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.983304 936 consensus_queue.cc:260] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:20.983690 936 ts_tablet_manager.cc:1398] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:20.983913 934 tablet_bootstrap.cc:492] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:20.984012 934 ts_tablet_manager.cc:1367] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.050s user 0.001s sys 0.000s
I20240502 07:59:20.984421 934 raft_consensus.cc:348] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:20.984524 934 raft_consensus.cc:374] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:20.984553 934 raft_consensus.cc:729] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:20.984663 934 consensus_queue.cc:260] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:20.984916 934 ts_tablet_manager.cc:1398] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:20.992681 836 consensus_queue.cc:1035] T 497e7c78a2274cc1a8406cdab125e0d3 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 07:59:21.019157 25019 consensus_peers.cc:485] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:21.030499 25213 consensus_queue.cc:237] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:21.031788 24952 raft_consensus.cc:1261] T a7da69fb73f94ce0b8c24599c8ae4d15 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:21.031842 22632 raft_consensus.cc:1261] T a7da69fb73f94ce0b8c24599c8ae4d15 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:21.032377 492 consensus_queue.cc:1035] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:21.032644 492 consensus_queue.cc:1035] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:21.034747 492 raft_consensus.cc:2932] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:21.035246 24952 raft_consensus.cc:2932] T a7da69fb73f94ce0b8c24599c8ae4d15 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:21.038211 22228 catalog_manager.cc:5484] T a7da69fb73f94ce0b8c24599c8ae4d15 P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:21.040298 22632 raft_consensus.cc:2932] T a7da69fb73f94ce0b8c24599c8ae4d15 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:21.042383 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet a7da69fb73f94ce0b8c24599c8ae4d15 with delete_type TABLET_DATA_TOMBSTONED (TS 227eec4b5f014d349661f6e0b39b9055 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:21.043066 365 consensus_queue.cc:1035] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:21.043491 969 tablet_replica.cc:330] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:21.043663 969 raft_consensus.cc:2229] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:21.043820 969 raft_consensus.cc:2258] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:21.046145 969 ts_tablet_manager.cc:1875] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:21.048641 969 ts_tablet_manager.cc:1888] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:21.048768 969 log.cc:1196] T a7da69fb73f94ce0b8c24599c8ae4d15 P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/a7da69fb73f94ce0b8c24599c8ae4d15
I20240502 07:59:21.049505 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet a7da69fb73f94ce0b8c24599c8ae4d15 (table rebalancer_extra_table_26 [id=e0a920fd5961477b92891b52ec936746]) successfully deleted
I20240502 07:59:21.060834 22229 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_60"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:21.067656 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 521231ade647437683f225bedbd8d06b (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=ce539bf5a9154312a1e9464d63785482]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:21.067786 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 1664c567ddad41acb99af72602afffa7 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=ce539bf5a9154312a1e9464d63785482]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:21.067881 22613 tablet_service.cc:1467] Processing CreateTablet for tablet 1664c567ddad41acb99af72602afffa7 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=ce539bf5a9154312a1e9464d63785482]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:21.067970 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 521231ade647437683f225bedbd8d06b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:21.068089 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1664c567ddad41acb99af72602afffa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:21.068141 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1664c567ddad41acb99af72602afffa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:21.070505 936 tablet_bootstrap.cc:492] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:21.070629 836 raft_consensus.cc:482] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:21.070714 836 raft_consensus.cc:504] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:21.071062 836 leader_election.cc:288] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:21.071390 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d8bd64c2c544c0ad14a4e89bb82d05" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:21.071538 25084 raft_consensus.cc:2454] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:21.071573 936 tablet_bootstrap.cc:654] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:21.071641 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d8bd64c2c544c0ad14a4e89bb82d05" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:21.071760 25213 raft_consensus.cc:2454] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:21.071875 22698 leader_election.cc:302] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:21.072059 836 raft_consensus.cc:2781] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:21.072111 836 raft_consensus.cc:482] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:21.072160 836 raft_consensus.cc:3037] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:21.072413 978 tablet_bootstrap.cc:492] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:21.073166 836 raft_consensus.cc:504] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:21.073580 836 leader_election.cc:288] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:21.073598 22351 tablet_service.cc:1467] Processing CreateTablet for tablet 52e8c7b14d4b45fa9f7b28e755e1f5df (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=ce539bf5a9154312a1e9464d63785482]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:21.073830 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52e8c7b14d4b45fa9f7b28e755e1f5df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:21.074535 24930 tablet_service.cc:1467] Processing CreateTablet for tablet 521231ade647437683f225bedbd8d06b (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=ce539bf5a9154312a1e9464d63785482]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:21.074815 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 521231ade647437683f225bedbd8d06b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:21.075197 936 tablet_bootstrap.cc:492] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:21.075282 936 ts_tablet_manager.cc:1367] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:21.075345 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d8bd64c2c544c0ad14a4e89bb82d05" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:21.075474 25084 raft_consensus.cc:3037] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:21.075863 936 raft_consensus.cc:348] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:21.075953 936 raft_consensus.cc:374] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:21.075984 936 raft_consensus.cc:729] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:21.076114 936 consensus_queue.cc:260] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:21.076246 22613 tablet_service.cc:1467] Processing CreateTablet for tablet 52e8c7b14d4b45fa9f7b28e755e1f5df (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=ce539bf5a9154312a1e9464d63785482]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:21.076469 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52e8c7b14d4b45fa9f7b28e755e1f5df. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:21.076501 936 ts_tablet_manager.cc:1398] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:21.076493 25084 raft_consensus.cc:2454] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:21.076689 936 tablet_bootstrap.cc:492] T 1664c567ddad41acb99af72602afffa7 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:21.076849 22698 leader_election.cc:302] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:21.077036 32718 raft_consensus.cc:2781] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:21.074553 24931 tablet_service.cc:1467] Processing CreateTablet for tablet 1664c567ddad41acb99af72602afffa7 (DEFAULT_TABLE table=rebalancer_extra_table_60 [id=ce539bf5a9154312a1e9464d63785482]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:21.077121 32718 raft_consensus.cc:686] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:59:21.077270 24931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1664c567ddad41acb99af72602afffa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:21.077272 32718 consensus_queue.cc:237] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:21.077795 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85d8bd64c2c544c0ad14a4e89bb82d05" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:21.077898 25213 raft_consensus.cc:3037] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:21.078913 25213 raft_consensus.cc:2454] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:21.079033 981 tablet_bootstrap.cc:492] T 521231ade647437683f225bedbd8d06b P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:21.079229 936 tablet_bootstrap.cc:654] T 1664c567ddad41acb99af72602afffa7 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:21.080173 981 tablet_bootstrap.cc:654] T 521231ade647437683f225bedbd8d06b P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:21.081512 981 tablet_bootstrap.cc:492] T 521231ade647437683f225bedbd8d06b P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:21.081595 981 ts_tablet_manager.cc:1367] T 521231ade647437683f225bedbd8d06b P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:21.081635 936 tablet_bootstrap.cc:492] T 1664c567ddad41acb99af72602afffa7 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:21.081707 936 ts_tablet_manager.cc:1367] T 1664c567ddad41acb99af72602afffa7 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:21.082082 981 raft_consensus.cc:348] T 521231ade647437683f225bedbd8d06b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:21.082190 981 raft_consensus.cc:374] T 521231ade647437683f225bedbd8d06b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:21.082177 936 raft_consensus.cc:348] T 1664c567ddad41acb99af72602afffa7 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:21.082243 981 raft_consensus.cc:729] T 521231ade647437683f225bedbd8d06b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:21.082273 936 raft_consensus.cc:374] T 1664c567ddad41acb99af72602afffa7 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:21.082310 936 raft_consensus.cc:729] T 1664c567ddad41acb99af72602afffa7 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:21.082387 981 consensus_queue.cc:260] T 521231ade647437683f225bedbd8d06b P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:21.082441 936 consensus_queue.cc:260] T 1664c567ddad41acb99af72602afffa7 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:21.082691 981 ts_tablet_manager.cc:1398] T 521231ade647437683f225bedbd8d06b P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:21.082706 936 ts_tablet_manager.cc:1398] T 1664c567ddad41acb99af72602afffa7 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:21.082864 981 tablet_bootstrap.cc:492] T 1664c567ddad41acb99af72602afffa7 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:21.083739 981 tablet_bootstrap.cc:654] T 1664c567ddad41acb99af72602afffa7 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
W20240502 07:59:21.085222 22216 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.085165 22228 catalog_manager.cc:5484] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:21.088189 978 tablet_bootstrap.cc:654] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
W20240502 07:59:21.088418 22216 catalog_manager.cc:4573] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): CreateTablet RPC failed for tablet 52e8c7b14d4b45fa9f7b28e755e1f5df: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.088958 980 tablet_bootstrap.cc:492] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:21.089754 981 tablet_bootstrap.cc:492] T 1664c567ddad41acb99af72602afffa7 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:21.089830 981 ts_tablet_manager.cc:1367] T 1664c567ddad41acb99af72602afffa7 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20240502 07:59:21.090198 981 raft_consensus.cc:348] T 1664c567ddad41acb99af72602afffa7 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:21.090307 981 raft_consensus.cc:374] T 1664c567ddad41acb99af72602afffa7 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:21.090350 981 raft_consensus.cc:729] T 1664c567ddad41acb99af72602afffa7 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:21.090493 981 consensus_queue.cc:260] T 1664c567ddad41acb99af72602afffa7 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:21.090782 981 ts_tablet_manager.cc:1398] T 1664c567ddad41acb99af72602afffa7 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:21.099423 980 tablet_bootstrap.cc:654] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:21.100664 22499 raft_consensus.cc:1203] T caf2dd9c96b94566b216692a9370f16e P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20240502 07:59:21.102342 980 tablet_bootstrap.cc:492] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:21.102523 980 ts_tablet_manager.cc:1367] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 07:59:21.102685 978 tablet_bootstrap.cc:492] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:21.102802 978 ts_tablet_manager.cc:1367] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20240502 07:59:21.103014 980 raft_consensus.cc:348] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:21.103113 980 raft_consensus.cc:374] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:21.103145 980 raft_consensus.cc:729] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:21.103292 980 consensus_queue.cc:260] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:21.103289 978 raft_consensus.cc:348] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:21.103469 978 raft_consensus.cc:374] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:21.103519 978 raft_consensus.cc:729] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:21.103672 978 consensus_queue.cc:260] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:21.104053 978 ts_tablet_manager.cc:1398] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:21.104543 980 ts_tablet_manager.cc:1398] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:21.107208 978 tablet_bootstrap.cc:492] T 52e8c7b14d4b45fa9f7b28e755e1f5df P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:21.110164 978 tablet_bootstrap.cc:654] T 52e8c7b14d4b45fa9f7b28e755e1f5df P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:21.111361 978 tablet_bootstrap.cc:492] T 52e8c7b14d4b45fa9f7b28e755e1f5df P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:21.111490 978 ts_tablet_manager.cc:1367] T 52e8c7b14d4b45fa9f7b28e755e1f5df P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:21.112278 978 raft_consensus.cc:348] T 52e8c7b14d4b45fa9f7b28e755e1f5df P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:21.112402 978 raft_consensus.cc:374] T 52e8c7b14d4b45fa9f7b28e755e1f5df P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:21.112432 978 raft_consensus.cc:729] T 52e8c7b14d4b45fa9f7b28e755e1f5df P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:21.112984 978 consensus_queue.cc:260] T 52e8c7b14d4b45fa9f7b28e755e1f5df P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:21.114120 978 ts_tablet_manager.cc:1398] T 52e8c7b14d4b45fa9f7b28e755e1f5df P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:21.126886 836 raft_consensus.cc:482] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:21.127027 836 raft_consensus.cc:504] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:21.127507 836 leader_election.cc:288] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:21.128114 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5034380b5c0846faa1494140d67203cc" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:21.129989 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5034380b5c0846faa1494140d67203cc" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:21.130466 22698 leader_election.cc:302] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014; no voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055
I20240502 07:59:21.130656 836 raft_consensus.cc:2726] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240502 07:59:21.131561 22700 leader_election.cc:334] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.147667 415 raft_consensus.cc:482] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:21.147801 415 raft_consensus.cc:504] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:21.148221 415 leader_election.cc:288] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:21.148911 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1664c567ddad41acb99af72602afffa7" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:21.148989 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1664c567ddad41acb99af72602afffa7" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:21.149080 24951 raft_consensus.cc:2454] T 1664c567ddad41acb99af72602afffa7 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:21.149116 22764 raft_consensus.cc:2454] T 1664c567ddad41acb99af72602afffa7 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:21.149489 22570 leader_election.cc:302] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:21.149825 415 raft_consensus.cc:2781] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:21.149892 415 raft_consensus.cc:482] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:21.149940 415 raft_consensus.cc:3037] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:21.151211 415 raft_consensus.cc:504] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:21.151554 415 leader_election.cc:288] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:21.152081 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1664c567ddad41acb99af72602afffa7" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:21.152199 22764 raft_consensus.cc:3037] T 1664c567ddad41acb99af72602afffa7 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:21.152220 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1664c567ddad41acb99af72602afffa7" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:21.152323 24951 raft_consensus.cc:3037] T 1664c567ddad41acb99af72602afffa7 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:21.153191 22764 raft_consensus.cc:2454] T 1664c567ddad41acb99af72602afffa7 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:21.153313 24951 raft_consensus.cc:2454] T 1664c567ddad41acb99af72602afffa7 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:21.153633 22570 leader_election.cc:302] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:21.155097 1024 raft_consensus.cc:2781] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:21.155265 1024 raft_consensus.cc:686] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:59:21.155508 1024 consensus_queue.cc:237] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:21.160279 22222 catalog_manager.cc:5484] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:21.167385 963 raft_consensus.cc:482] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.167519 963 raft_consensus.cc:504] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:21.168138 963 leader_election.cc:288] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:21.168414 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05911d56b0344eb8a53aba5f4bf0170c" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
W20240502 07:59:21.169253 25151 leader_election.cc:341] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): Not found: Tablet not found: 05911d56b0344eb8a53aba5f4bf0170c
I20240502 07:59:21.169354 25151 leader_election.cc:302] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432; no voters: 227eec4b5f014d349661f6e0b39b9055, b533a7d8d97943609c6e28e3e36b78a4
I20240502 07:59:21.169582 963 raft_consensus.cc:2726] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240502 07:59:21.169701 25150 leader_election.cc:334] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.170536 32660 raft_consensus.cc:482] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.170629 32660 raft_consensus.cc:504] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:21.171046 32660 leader_election.cc:288] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:21.171469 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05911d56b0344eb8a53aba5f4bf0170c" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:21.171581 25213 raft_consensus.cc:2454] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
W20240502 07:59:21.171908 25020 leader_election.cc:341] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): Not found: Tablet not found: 05911d56b0344eb8a53aba5f4bf0170c
W20240502 07:59:21.172343 25019 leader_election.cc:334] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.172415 25019 leader_election.cc:302] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters: 4b4e7eeb7f3f42119c0363a79febc54b, b533a7d8d97943609c6e28e3e36b78a4
I20240502 07:59:21.172545 32660 raft_consensus.cc:2726] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 07:59:21.196429 1024 raft_consensus.cc:482] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.196589 1024 raft_consensus.cc:504] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:21.197022 1024 leader_election.cc:288] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.197288 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ae14d069e048ccba7fdfc79951cec0" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
W20240502 07:59:21.199692 22569 leader_election.cc:334] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.199791 22569 leader_election.cc:302] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f; no voters: 4b4e7eeb7f3f42119c0363a79febc54b, b533a7d8d97943609c6e28e3e36b78a4
I20240502 07:59:21.200117 415 raft_consensus.cc:2726] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 07:59:21.211494 22228 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:21.215133 25213 raft_consensus.cc:1261] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:21.215149 25083 raft_consensus.cc:1261] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:21.215505 836 consensus_queue.cc:1035] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:21.215628 31905 consensus_queue.cc:1035] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:21.218712 22228 catalog_manager.cc:5576] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591) does not have the latest schema for tablet 85d8bd64c2c544c0ad14a4e89bb82d05 (table rebalancer_dynamic_table [id=daee501832934f049a243f2024abda3d]). Expected version 1 got 0
I20240502 07:59:21.219447 22224 catalog_manager.cc:5576] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733) does not have the latest schema for tablet 85d8bd64c2c544c0ad14a4e89bb82d05 (table rebalancer_dynamic_table [id=daee501832934f049a243f2024abda3d]). Expected version 1 got 0
I20240502 07:59:21.219853 1028 tablet.cc:1722] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:21.220230 1028 tablet.cc:1620] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:59:21.221165 1029 tablet.cc:1722] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:21.221539 1029 tablet.cc:1620] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:21.223454 1030 tablet.cc:1722] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:21.223793 1030 tablet.cc:1620] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:21.225239 32390 raft_consensus.cc:482] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:21.225359 32390 raft_consensus.cc:504] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:21.225754 32390 leader_election.cc:288] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.226003 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52e8c7b14d4b45fa9f7b28e755e1f5df" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:21.226125 22634 raft_consensus.cc:2454] T 52e8c7b14d4b45fa9f7b28e755e1f5df P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:21.226400 22308 leader_election.cc:302] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:21.226598 308 raft_consensus.cc:2781] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:21.226660 308 raft_consensus.cc:482] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:21.226711 308 raft_consensus.cc:3037] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
W20240502 07:59:21.227306 22307 leader_election.cc:334] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.227521 22228 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=daee501832934f049a243f2024abda3d] alter complete (version 1)
I20240502 07:59:21.227710 308 raft_consensus.cc:504] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:21.227999 308 leader_election.cc:288] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
W20240502 07:59:21.229532 22307 leader_election.cc:334] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.233479 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52e8c7b14d4b45fa9f7b28e755e1f5df" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:21.233614 22634 raft_consensus.cc:3037] T 52e8c7b14d4b45fa9f7b28e755e1f5df P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:21.234580 22634 raft_consensus.cc:2454] T 52e8c7b14d4b45fa9f7b28e755e1f5df P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:21.235031 22308 leader_election.cc:302] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters: 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:21.235368 308 raft_consensus.cc:2781] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:21.235463 308 raft_consensus.cc:686] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:59:21.235613 308 consensus_queue.cc:237] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
W20240502 07:59:21.236913 22700 consensus_peers.cc:462] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.239842 22224 catalog_manager.cc:5484] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
W20240502 07:59:21.249874 22569 consensus_peers.cc:462] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:21.270509 25150 consensus_peers.cc:462] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.301668 963 raft_consensus.cc:482] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.301791 963 raft_consensus.cc:504] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:21.302227 963 leader_election.cc:288] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.302551 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cd6c14c7c8b4ad29f2107e94fd11acc" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
W20240502 07:59:21.303239 25150 leader_election.cc:334] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.303298 25150 leader_election.cc:302] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432; no voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:21.303431 963 raft_consensus.cc:2726] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 07:59:21.331863 836 raft_consensus.cc:482] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.331988 836 raft_consensus.cc:504] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:21.332382 836 leader_election.cc:288] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.332746 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f916726456f4aa8936a9366468bc471" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
W20240502 07:59:21.333397 22700 leader_election.cc:334] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.333460 22700 leader_election.cc:302] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014; no voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:21.333585 836 raft_consensus.cc:2726] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 07:59:21.336099 22224 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:21.338699 836 raft_consensus.cc:482] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:21.338804 836 raft_consensus.cc:504] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:21.339129 836 leader_election.cc:288] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:21.339715 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "521231ade647437683f225bedbd8d06b" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:21.339869 24951 raft_consensus.cc:2454] T 521231ade647437683f225bedbd8d06b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
W20240502 07:59:21.340199 22700 leader_election.cc:334] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.340634 22701 leader_election.cc:302] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1aecef81733c47b3b468504934357231, f00f3a1876444ad2bf9c919068ab8014; no voters: 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:21.340828 31905 raft_consensus.cc:2781] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:21.340893 31905 raft_consensus.cc:482] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:21.340922 31905 raft_consensus.cc:3037] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:21.342154 31905 raft_consensus.cc:504] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:21.342509 31905 leader_election.cc:288] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:21.342960 1028 tablet.cc:1722] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:21.343518 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "521231ade647437683f225bedbd8d06b" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:21.343557 601 raft_consensus.cc:482] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.343641 24952 raft_consensus.cc:3037] T 521231ade647437683f225bedbd8d06b P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:21.343663 601 raft_consensus.cc:504] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:21.343873 1028 tablet.cc:1620] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:59:21.344245 601 leader_election.cc:288] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
W20240502 07:59:21.344481 22700 leader_election.cc:334] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.344553 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751f8c9cbf684b01b90e7bd26685f8bf" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:21.344709 24952 raft_consensus.cc:2454] T 521231ade647437683f225bedbd8d06b P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:21.345310 22701 leader_election.cc:302] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1aecef81733c47b3b468504934357231, f00f3a1876444ad2bf9c919068ab8014; no voters: 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:21.345502 31905 raft_consensus.cc:2781] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:21.345585 31905 raft_consensus.cc:686] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
W20240502 07:59:21.345685 24887 leader_election.cc:334] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.345741 24887 leader_election.cc:302] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1aecef81733c47b3b468504934357231; no voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:21.345714 31905 consensus_queue.cc:237] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:21.345852 32485 raft_consensus.cc:2726] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 07:59:21.346866 1030 tablet.cc:1722] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:21.347497 1029 tablet.cc:1722] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:21.347811 1029 tablet.cc:1620] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:21.347811 1030 tablet.cc:1620] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:21.350128 22224 catalog_manager.cc:5484] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:21.353142 22223 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=daee501832934f049a243f2024abda3d] alter complete (version 2)
I20240502 07:59:21.364125 32485 raft_consensus.cc:482] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.364236 32485 raft_consensus.cc:504] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:21.364699 32485 leader_election.cc:288] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:21.365226 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5034380b5c0846faa1494140d67203cc" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:21.365367 32390 raft_consensus.cc:482] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.365450 32390 raft_consensus.cc:504] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:21.365818 32390 leader_election.cc:288] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
W20240502 07:59:21.366101 24887 leader_election.cc:334] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.366192 24885 leader_election.cc:302] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1aecef81733c47b3b468504934357231; no voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:21.366333 32485 raft_consensus.cc:2726] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 07:59:21.366600 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d9a73be1fd472db39860e538b9c97e" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:21.366652 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5034380b5c0846faa1494140d67203cc" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:21.366766 22763 raft_consensus.cc:2454] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
W20240502 07:59:21.368995 22307 leader_election.cc:334] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.369069 22307 leader_election.cc:302] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4; no voters: 4b4e7eeb7f3f42119c0363a79febc54b, f00f3a1876444ad2bf9c919068ab8014
I20240502 07:59:21.369228 32390 raft_consensus.cc:2726] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 07:59:21.374850 25193 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:21.375151 24931 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:21.385499 994 raft_consensus.cc:482] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.385628 994 raft_consensus.cc:504] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:21.386047 994 leader_election.cc:288] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
W20240502 07:59:21.387334 22438 leader_election.cc:334] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.390414 25063 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:21.390595 22351 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:21.390796 22613 tablet_service.cc:1429] Tablet server has 23 leaders and 0 scanners
I20240502 07:59:21.390867 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d1511fc27c24820afaa6e41161be8bb" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:21.391263 22439 leader_election.cc:302] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fabddb16c85944648a710ec52fc7244c; no voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:21.391515 994 raft_consensus.cc:2726] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 07:59:21.398216 22745 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:21.438570 22481 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:21.455746 22228 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:21.463629 1028 tablet.cc:1722] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:21.466656 1030 tablet.cc:1722] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:21.468099 1029 tablet.cc:1722] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:21.469909 22228 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=daee501832934f049a243f2024abda3d] alter complete (version 3)
I20240502 07:59:21.480203 963 raft_consensus.cc:482] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.480314 963 raft_consensus.cc:504] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:21.480684 963 leader_election.cc:288] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.481217 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d1511fc27c24820afaa6e41161be8bb" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:21.481382 22501 raft_consensus.cc:2454] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:21.481379 31905 raft_consensus.cc:482] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.481484 31905 raft_consensus.cc:504] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:21.481747 25148 leader_election.cc:302] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:21.481848 31905 leader_election.cc:288] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:21.482053 963 raft_consensus.cc:2781] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 07:59:21.482105 963 raft_consensus.cc:482] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.482143 963 raft_consensus.cc:3037] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.482245 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf443ada59854e6da532aa5832a9a564" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:21.483350 963 raft_consensus.cc:504] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
W20240502 07:59:21.483376 25150 leader_election.cc:334] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.483691 963 leader_election.cc:288] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.484699 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d1511fc27c24820afaa6e41161be8bb" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:21.484810 22501 raft_consensus.cc:3037] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.485900 22501 raft_consensus.cc:2454] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 2.
I20240502 07:59:21.486240 25148 leader_election.cc:302] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, fabddb16c85944648a710ec52fc7244c; no voters:
W20240502 07:59:21.487280 22700 leader_election.cc:334] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.487354 22700 leader_election.cc:302] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014; no voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:21.487488 836 raft_consensus.cc:2726] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 07:59:21.487756 963 raft_consensus.cc:2781] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:21.487841 963 raft_consensus.cc:686] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [term 2 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:21.488006 963 consensus_queue.cc:237] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
W20240502 07:59:21.489352 25150 leader_election.cc:334] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.490710 22223 catalog_manager.cc:5484] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 1 to 2, leader changed from 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 2 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:21.492070 32660 raft_consensus.cc:482] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.492192 32660 raft_consensus.cc:504] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:21.492568 32660 leader_election.cc:288] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:21.493072 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5034380b5c0846faa1494140d67203cc" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:21.493238 24951 raft_consensus.cc:2454] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
W20240502 07:59:21.493839 25019 leader_election.cc:334] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.494287 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5034380b5c0846faa1494140d67203cc" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:21.494434 22763 raft_consensus.cc:2454] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:21.494865 25020 leader_election.cc:302] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055, f00f3a1876444ad2bf9c919068ab8014; no voters: 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:21.495020 32660 raft_consensus.cc:2781] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 07:59:21.495079 32660 raft_consensus.cc:482] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.495110 32660 raft_consensus.cc:3037] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.495891 32660 raft_consensus.cc:504] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:21.496284 32660 leader_election.cc:288] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:21.496928 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5034380b5c0846faa1494140d67203cc" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:21.496928 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5034380b5c0846faa1494140d67203cc" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:21.497035 22763 raft_consensus.cc:3037] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.497035 24951 raft_consensus.cc:3037] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Advancing to term 2
W20240502 07:59:21.497756 25019 leader_election.cc:334] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.498013 22763 raft_consensus.cc:2454] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 2.
I20240502 07:59:21.498013 24951 raft_consensus.cc:2454] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 2.
I20240502 07:59:21.498328 25020 leader_election.cc:302] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055, f00f3a1876444ad2bf9c919068ab8014; no voters: 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:21.498529 32660 raft_consensus.cc:2781] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:21.498600 32660 raft_consensus.cc:686] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [term 2 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:21.498733 32660 consensus_queue.cc:237] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:21.499073 365 raft_consensus.cc:482] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.499151 365 raft_consensus.cc:504] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:21.499456 365 leader_election.cc:288] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.500257 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "712a88cd545745e8937d295266e85f04" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
W20240502 07:59:21.501020 25019 leader_election.cc:334] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.501957 22223 catalog_manager.cc:5484] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 1 to 2, leader changed from 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 2 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:21.502215 25020 leader_election.cc:302] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055; no voters: 43930dee0cd5498fa8e357a34dd71432, 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:21.502413 32660 raft_consensus.cc:2726] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 07:59:21.512075 500 raft_consensus.cc:482] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.512230 500 raft_consensus.cc:504] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:21.512655 500 leader_election.cc:288] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.513185 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "712a88cd545745e8937d295266e85f04" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:21.513335 25084 raft_consensus.cc:2454] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:21.513680 25148 leader_election.cc:302] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
W20240502 07:59:21.514087 25150 leader_election.cc:334] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.514146 500 raft_consensus.cc:2781] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 07:59:21.514209 500 raft_consensus.cc:482] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.514256 500 raft_consensus.cc:3037] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.514664 601 raft_consensus.cc:482] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.514778 601 raft_consensus.cc:504] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:21.515161 601 leader_election.cc:288] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:21.515547 500 raft_consensus.cc:504] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:21.515807 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fcb3fef170343d494d8dfbba33052d9" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:21.515995 500 leader_election.cc:288] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.516165 308 raft_consensus.cc:482] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.516264 308 raft_consensus.cc:504] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
W20240502 07:59:21.516341 24887 leader_election.cc:334] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.516386 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "712a88cd545745e8937d295266e85f04" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:21.516418 24887 leader_election.cc:302] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1aecef81733c47b3b468504934357231; no voters: 227eec4b5f014d349661f6e0b39b9055, 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:21.516472 25084 raft_consensus.cc:3037] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.516572 601 raft_consensus.cc:2726] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 07:59:21.516671 308 leader_election.cc:288] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:21.517535 25084 raft_consensus.cc:2454] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 2.
I20240502 07:59:21.517753 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ca68d8eec874332b74df02eb55e0ff9" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
W20240502 07:59:21.518663 22307 leader_election.cc:334] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.518725 22307 leader_election.cc:302] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4; no voters: 4b4e7eeb7f3f42119c0363a79febc54b, e51b10af49484aaa8f0267aebc1bf27f
I20240502 07:59:21.518851 308 raft_consensus.cc:2726] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 07:59:21.519002 25148 leader_election.cc:302] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:21.519187 500 raft_consensus.cc:2781] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [term 2 FOLLOWER]: Leader election won for term 2
W20240502 07:59:21.519409 25150 leader_election.cc:334] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.519623 500 raft_consensus.cc:686] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [term 2 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:21.519766 500 consensus_queue.cc:237] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:21.522135 22223 catalog_manager.cc:5484] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 1 to 2, leader changed from 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 2 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:21.527577 500 raft_consensus.cc:482] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.527680 500 raft_consensus.cc:504] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:21.528017 500 leader_election.cc:288] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:21.528319 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2abf00f982e450fbe0124226789324a" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:21.528745 32660 raft_consensus.cc:482] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.528834 32660 raft_consensus.cc:504] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:21.529153 32660 leader_election.cc:288] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
W20240502 07:59:21.529453 25150 leader_election.cc:334] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.529520 25150 leader_election.cc:302] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432; no voters: 4b4e7eeb7f3f42119c0363a79febc54b, b533a7d8d97943609c6e28e3e36b78a4
I20240502 07:59:21.529649 500 raft_consensus.cc:2726] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 07:59:21.529678 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fcb3fef170343d494d8dfbba33052d9" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:21.529803 24951 raft_consensus.cc:2454] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:21.530092 25020 leader_election.cc:302] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
W20240502 07:59:21.530187 25019 leader_election.cc:334] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.530257 32660 raft_consensus.cc:2781] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 07:59:21.530323 32660 raft_consensus.cc:482] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.530365 32660 raft_consensus.cc:3037] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.531378 32660 raft_consensus.cc:504] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:21.531736 32660 leader_election.cc:288] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:21.532209 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fcb3fef170343d494d8dfbba33052d9" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:21.532320 24951 raft_consensus.cc:3037] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.533246 24951 raft_consensus.cc:2454] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 2.
I20240502 07:59:21.533532 25020 leader_election.cc:302] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
I20240502 07:59:21.533680 32660 raft_consensus.cc:2781] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:21.533737 32660 raft_consensus.cc:686] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [term 2 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:21.533864 32660 consensus_queue.cc:237] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
W20240502 07:59:21.533741 25019 leader_election.cc:334] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.536368 22223 catalog_manager.cc:5484] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 1 to 2, leader changed from 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 2 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:21.538833 1021 raft_consensus.cc:482] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.538925 1021 raft_consensus.cc:504] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:21.539243 1021 leader_election.cc:288] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.539733 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cd6c14c7c8b4ad29f2107e94fd11acc" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:21.539862 25213 raft_consensus.cc:2454] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:21.540150 25020 leader_election.cc:302] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:21.540309 1021 raft_consensus.cc:2781] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 07:59:21.540366 1021 raft_consensus.cc:482] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.540416 1021 raft_consensus.cc:3037] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.541222 1021 raft_consensus.cc:504] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:21.541600 1021 leader_election.cc:288] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.541914 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cd6c14c7c8b4ad29f2107e94fd11acc" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:21.542021 25213 raft_consensus.cc:3037] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Advancing to term 2
W20240502 07:59:21.543226 25019 leader_election.cc:334] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.543319 25213 raft_consensus.cc:2454] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 2.
W20240502 07:59:21.543394 25019 leader_election.cc:334] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.543740 25020 leader_election.cc:302] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters: 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:21.544092 1021 raft_consensus.cc:2781] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:21.544206 1021 raft_consensus.cc:686] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [term 2 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:21.544340 1021 consensus_queue.cc:237] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:21.546625 22223 catalog_manager.cc:5484] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 1 to 2, leader changed from 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 2 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:21.571720 22226 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:21.571875 22226 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:21.573978 22226 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 85d8bd64c2c544c0ad14a4e89bb82d05
I20240502 07:59:21.574654 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet 85d8bd64c2c544c0ad14a4e89bb82d05 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:21 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:21.574901 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet 85d8bd64c2c544c0ad14a4e89bb82d05 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:21 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:21.575503 25193 tablet_service.cc:1514] Processing DeleteTablet for tablet 85d8bd64c2c544c0ad14a4e89bb82d05 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:21 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:21.576522 1090 tablet_replica.cc:330] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:21.576606 1091 tablet_replica.cc:330] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:21.576727 1090 raft_consensus.cc:2229] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:21.576745 1091 raft_consensus.cc:2229] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:21.576850 1090 raft_consensus.cc:2258] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:21.576937 1091 raft_consensus.cc:2258] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:21.577235 1092 tablet_replica.cc:330] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:21.577510 1092 raft_consensus.cc:2229] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:21.577657 1091 ts_tablet_manager.cc:1875] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:21.577687 1092 raft_consensus.cc:2258] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:21.578231 1092 ts_tablet_manager.cc:1875] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:21.580169 1091 ts_tablet_manager.cc:1888] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:21.580307 1091 log.cc:1196] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/85d8bd64c2c544c0ad14a4e89bb82d05
I20240502 07:59:21.580860 1091 ts_tablet_manager.cc:1909] T 85d8bd64c2c544c0ad14a4e89bb82d05 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:59:21.581099 1092 ts_tablet_manager.cc:1888] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:21.581246 1092 log.cc:1196] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/85d8bd64c2c544c0ad14a4e89bb82d05
I20240502 07:59:21.581686 1092 ts_tablet_manager.cc:1909] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:59:21.577804 1090 ts_tablet_manager.cc:1875] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:21.583864 1090 ts_tablet_manager.cc:1888] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:21.583973 1090 log.cc:1196] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/85d8bd64c2c544c0ad14a4e89bb82d05
I20240502 07:59:21.584232 1090 ts_tablet_manager.cc:1909] T 85d8bd64c2c544c0ad14a4e89bb82d05 P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:59:21.584784 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet 85d8bd64c2c544c0ad14a4e89bb82d05 (table rebalancer_dynamic_table_new_name [id=daee501832934f049a243f2024abda3d]) successfully deleted
I20240502 07:59:21.585780 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 85d8bd64c2c544c0ad14a4e89bb82d05 (table rebalancer_dynamic_table_new_name [id=daee501832934f049a243f2024abda3d]) successfully deleted
I20240502 07:59:21.585933 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 85d8bd64c2c544c0ad14a4e89bb82d05 (table rebalancer_dynamic_table_new_name [id=daee501832934f049a243f2024abda3d]) successfully deleted
I20240502 07:59:21.587220 836 raft_consensus.cc:482] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.587321 836 raft_consensus.cc:504] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:21.587698 836 leader_election.cc:288] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:21.588050 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d9a73be1fd472db39860e538b9c97e" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:21.588194 22370 raft_consensus.cc:2454] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:21.588980 22701 leader_election.cc:302] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:21.589428 836 raft_consensus.cc:2781] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 07:59:21.589486 836 raft_consensus.cc:482] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.589540 836 raft_consensus.cc:3037] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.590629 500 raft_consensus.cc:482] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.590715 500 raft_consensus.cc:504] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:21.590710 836 raft_consensus.cc:504] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:21.591075 836 leader_election.cc:288] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 2 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:21.591109 500 leader_election.cc:288] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.591454 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79a67cfa3c554f2d85a520bb8f5ccc81" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:21.592260 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d9a73be1fd472db39860e538b9c97e" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:21.592371 22370 raft_consensus.cc:3037] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Advancing to term 2
W20240502 07:59:21.592526 25150 leader_election.cc:334] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.592583 25150 leader_election.cc:302] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432; no voters: 4b4e7eeb7f3f42119c0363a79febc54b, fabddb16c85944648a710ec52fc7244c
W20240502 07:59:21.592944 22700 leader_election.cc:334] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.593444 22370 raft_consensus.cc:2454] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 2.
W20240502 07:59:21.593472 22700 leader_election.cc:334] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.593760 492 raft_consensus.cc:2726] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 07:59:21.594084 22701 leader_election.cc:302] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, f00f3a1876444ad2bf9c919068ab8014; no voters: 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:21.596558 836 raft_consensus.cc:2781] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:21.596650 836 raft_consensus.cc:686] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [term 2 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:59:21.596786 836 consensus_queue.cc:237] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:21.599236 22226 catalog_manager.cc:5484] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 1 to 2, leader changed from 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 2 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:21.602784 1024 raft_consensus.cc:482] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.602896 1024 raft_consensus.cc:504] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:21.603302 1024 leader_election.cc:288] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.603935 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ca68d8eec874332b74df02eb55e0ff9" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:21.604063 22372 raft_consensus.cc:2454] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:21.604389 22570 leader_election.cc:302] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
W20240502 07:59:21.604496 22569 leader_election.cc:334] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.604573 1024 raft_consensus.cc:2781] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 07:59:21.604614 1024 raft_consensus.cc:482] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.604645 1024 raft_consensus.cc:3037] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.605751 1024 raft_consensus.cc:504] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:21.606068 1024 leader_election.cc:288] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 2 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.607173 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ca68d8eec874332b74df02eb55e0ff9" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:21.607295 22372 raft_consensus.cc:3037] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.608297 22372 raft_consensus.cc:2454] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 2.
W20240502 07:59:21.608857 22569 leader_election.cc:334] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.609591 22570 leader_election.cc:302] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters: 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:21.609844 1024 raft_consensus.cc:2781] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:21.609920 1024 raft_consensus.cc:686] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [term 2 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:59:21.610044 1024 consensus_queue.cc:237] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:21.612926 22224 catalog_manager.cc:5484] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 1 to 2, leader changed from 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 2 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:21.624527 994 raft_consensus.cc:482] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.624647 994 raft_consensus.cc:504] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:21.625044 994 leader_election.cc:288] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.625919 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79a67cfa3c554f2d85a520bb8f5ccc81" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:21.626050 25213 raft_consensus.cc:2454] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:21.626304 22439 leader_election.cc:302] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:21.627252 994 raft_consensus.cc:2781] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 07:59:21.627308 994 raft_consensus.cc:482] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.627350 994 raft_consensus.cc:3037] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.628490 994 raft_consensus.cc:504] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:21.628818 994 leader_election.cc:288] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 2 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.628930 308 raft_consensus.cc:482] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.629022 308 raft_consensus.cc:504] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:21.629423 308 leader_election.cc:288] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.630102 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2abf00f982e450fbe0124226789324a" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:21.630199 25213 raft_consensus.cc:2454] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:21.630491 22308 leader_election.cc:302] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:21.630811 308 raft_consensus.cc:2781] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 07:59:21.630869 308 raft_consensus.cc:482] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.630913 308 raft_consensus.cc:3037] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Advancing to term 2
W20240502 07:59:21.631210 22307 leader_election.cc:334] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.631963 308 raft_consensus.cc:504] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:21.632290 308 leader_election.cc:288] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 2 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
W20240502 07:59:21.633001 22438 leader_election.cc:334] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.633524 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79a67cfa3c554f2d85a520bb8f5ccc81" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:21.633634 25213 raft_consensus.cc:3037] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.632200 1024 consensus_queue.cc:1035] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:21.634522 25213 raft_consensus.cc:2454] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 2.
I20240502 07:59:21.634872 22439 leader_election.cc:302] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, fabddb16c85944648a710ec52fc7244c; no voters:
W20240502 07:59:21.635963 22438 leader_election.cc:334] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:21.636749 22307 leader_election.cc:334] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.636977 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2abf00f982e450fbe0124226789324a" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:21.637075 25213 raft_consensus.cc:3037] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.638031 25213 raft_consensus.cc:2454] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 2.
I20240502 07:59:21.638293 22308 leader_election.cc:302] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, b533a7d8d97943609c6e28e3e36b78a4; no voters: 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:21.638444 308 raft_consensus.cc:2781] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:21.638522 308 raft_consensus.cc:686] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [term 2 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:59:21.638592 994 raft_consensus.cc:2781] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:21.638602 32698 consensus_queue.cc:1035] T 1664c567ddad41acb99af72602afffa7 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:21.638666 994 raft_consensus.cc:686] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [term 2 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:59:21.638656 308 consensus_queue.cc:237] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:21.638783 994 consensus_queue.cc:237] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:21.641218 22227 catalog_manager.cc:5484] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 1 to 2, leader changed from 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 2 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:21.643143 22222 catalog_manager.cc:5484] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 1 to 2, leader changed from 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 2 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:21.645905 32390 consensus_queue.cc:1035] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 07:59:21.651327 22307 consensus_peers.cc:485] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:21.656241 32390 raft_consensus.cc:482] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.656363 32390 raft_consensus.cc:504] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:21.656746 32390 leader_election.cc:288] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.657435 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ae14d069e048ccba7fdfc79951cec0" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:21.657568 22634 raft_consensus.cc:2454] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
W20240502 07:59:21.657756 22307 leader_election.cc:334] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.659276 22308 leader_election.cc:302] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters: 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:21.659582 308 raft_consensus.cc:2781] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 07:59:21.659634 308 raft_consensus.cc:482] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.659670 308 raft_consensus.cc:3037] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.660708 308 raft_consensus.cc:504] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
I20240502 07:59:21.661046 308 leader_election.cc:288] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 2 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.661931 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ae14d069e048ccba7fdfc79951cec0" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:21.662065 22634 raft_consensus.cc:3037] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.663172 22634 raft_consensus.cc:2454] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 2.
I20240502 07:59:21.663439 1021 raft_consensus.cc:482] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.663519 1021 raft_consensus.cc:504] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:21.663815 1021 leader_election.cc:288] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:21.664397 22308 leader_election.cc:302] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:21.664579 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751f8c9cbf684b01b90e7bd26685f8bf" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:21.664691 24952 raft_consensus.cc:2454] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:21.665251 25020 leader_election.cc:302] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
I20240502 07:59:21.666261 1021 raft_consensus.cc:2781] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 07:59:21.666321 1021 raft_consensus.cc:482] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.666358 1021 raft_consensus.cc:3037] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.667337 1021 raft_consensus.cc:504] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:21.667629 1021 leader_election.cc:288] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:21.669028 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751f8c9cbf684b01b90e7bd26685f8bf" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:21.669147 24952 raft_consensus.cc:3037] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.670140 24952 raft_consensus.cc:2454] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 2.
I20240502 07:59:21.670562 25020 leader_election.cc:302] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
W20240502 07:59:21.670802 25019 leader_election.cc:334] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.671413 32660 raft_consensus.cc:2781] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:21.671521 32660 raft_consensus.cc:686] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [term 2 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:21.671651 32660 consensus_queue.cc:237] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:21.672513 308 raft_consensus.cc:2781] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:21.672602 308 raft_consensus.cc:686] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [term 2 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:59:21.672724 308 consensus_queue.cc:237] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } }
W20240502 07:59:21.673403 22307 leader_election.cc:334] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.675570 22226 catalog_manager.cc:5484] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 1 to 2, leader changed from 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 2 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } }
I20240502 07:59:21.676096 22222 catalog_manager.cc:5484] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 1 to 2, leader changed from 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 2 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:21.676756 22227 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:21.679076 24931 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:21.682466 25063 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
W20240502 07:59:21.686136 25019 leader_election.cc:334] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.686473 32660 raft_consensus.cc:482] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.686573 32660 raft_consensus.cc:504] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:21.686914 32660 leader_election.cc:288] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.686992 32660 raft_consensus.cc:482] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.687055 32660 raft_consensus.cc:504] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:21.687386 32660 leader_election.cc:288] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:21.687808 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f916726456f4aa8936a9366468bc471" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:21.687932 22764 raft_consensus.cc:2454] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:21.688133 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf443ada59854e6da532aa5832a9a564" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:21.688239 22763 raft_consensus.cc:2454] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:21.688441 25020 leader_election.cc:302] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:21.688623 25020 leader_election.cc:302] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:21.688760 365 raft_consensus.cc:2781] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 07:59:21.688803 365 raft_consensus.cc:482] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.688846 365 raft_consensus.cc:3037] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.689893 365 raft_consensus.cc:504] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:21.690215 365 leader_election.cc:288] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:21.690332 365 raft_consensus.cc:2781] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240502 07:59:21.690376 365 raft_consensus.cc:482] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:21.690413 365 raft_consensus.cc:3037] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.691385 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f916726456f4aa8936a9366468bc471" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:21.691500 22763 raft_consensus.cc:3037] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.692605 22763 raft_consensus.cc:2454] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 2.
I20240502 07:59:21.692934 25020 leader_election.cc:302] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:21.693107 32660 raft_consensus.cc:2781] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:21.693202 32660 raft_consensus.cc:686] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [term 2 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:21.693369 32660 consensus_queue.cc:237] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:21.695068 25063 tablet_service.cc:1467] Processing CreateTablet for tablet df3b695402f3496f8ecf31d952ae738d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b2d5de90f1924f84b4163927d7836158]), partition=
I20240502 07:59:21.695108 25193 tablet_service.cc:1467] Processing CreateTablet for tablet df3b695402f3496f8ecf31d952ae738d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b2d5de90f1924f84b4163927d7836158]), partition=
I20240502 07:59:21.695302 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df3b695402f3496f8ecf31d952ae738d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:21.695338 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df3b695402f3496f8ecf31d952ae738d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:21.695510 22351 tablet_service.cc:1467] Processing CreateTablet for tablet df3b695402f3496f8ecf31d952ae738d (DEFAULT_TABLE table=rebalancer_dynamic_table [id=b2d5de90f1924f84b4163927d7836158]), partition=
I20240502 07:59:21.695721 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df3b695402f3496f8ecf31d952ae738d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:21.695691 365 raft_consensus.cc:504] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:21.695972 365 leader_election.cc:288] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:21.698711 22227 catalog_manager.cc:5484] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 1 to 2, leader changed from 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 2 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:21.699061 1119 tablet_bootstrap.cc:492] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:21.700857 1119 tablet_bootstrap.cc:654] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:21.700876 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf443ada59854e6da532aa5832a9a564" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:21.700986 22763 raft_consensus.cc:3037] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:21.701944 22763 raft_consensus.cc:2454] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 2.
I20240502 07:59:21.702278 25020 leader_election.cc:302] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:21.702591 365 raft_consensus.cc:2781] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:21.702684 365 raft_consensus.cc:686] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [term 2 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:21.702811 365 consensus_queue.cc:237] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:21.702970 1119 tablet_bootstrap.cc:492] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:21.703080 1119 ts_tablet_manager.cc:1367] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:21.703362 1120 tablet_bootstrap.cc:492] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:21.704367 1120 tablet_bootstrap.cc:654] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:21.705603 1119 raft_consensus.cc:348] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:21.705657 22227 catalog_manager.cc:5484] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 1 to 2, leader changed from 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133) to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 2 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:21.705785 1119 raft_consensus.cc:374] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:21.705917 1119 raft_consensus.cc:729] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:21.706085 1119 consensus_queue.cc:260] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:21.706512 1119 ts_tablet_manager.cc:1398] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:21.708602 22745 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:21.709195 25193 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:21.709782 1124 tablet_bootstrap.cc:492] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:21.709962 22481 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 07:59:21.710891 1124 tablet_bootstrap.cc:654] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
W20240502 07:59:21.711999 25019 leader_election.cc:334] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.712097 1124 tablet_bootstrap.cc:492] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
W20240502 07:59:21.712153 25019 leader_election.cc:334] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.712172 1124 ts_tablet_manager.cc:1367] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
W20240502 07:59:21.712237 25019 leader_election.cc:334] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:21.712311 25019 leader_election.cc:334] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.712793 1124 raft_consensus.cc:348] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:21.712894 1124 raft_consensus.cc:374] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:21.712922 1124 raft_consensus.cc:729] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:21.713042 1124 consensus_queue.cc:260] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:21.713348 1124 ts_tablet_manager.cc:1398] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:21.719594 22351 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:21.749295 31905 consensus_queue.cc:1035] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:21.769460 1120 tablet_bootstrap.cc:492] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:21.769584 1120 ts_tablet_manager.cc:1367] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.066s user 0.001s sys 0.000s
I20240502 07:59:21.770046 1120 raft_consensus.cc:348] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:21.770164 1120 raft_consensus.cc:374] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:21.770192 1120 raft_consensus.cc:729] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:21.770324 1120 consensus_queue.cc:260] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:21.770622 1120 ts_tablet_manager.cc:1398] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:21.771001 22613 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
W20240502 07:59:21.775573 22700 consensus_peers.cc:485] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:21.803841 500 raft_consensus.cc:482] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:21.803966 500 raft_consensus.cc:504] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
W20240502 07:59:21.804263 22569 consensus_peers.cc:462] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:21.804373 500 leader_election.cc:288] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:21.804900 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df3b695402f3496f8ecf31d952ae738d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:21.804946 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df3b695402f3496f8ecf31d952ae738d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:21.805049 22372 raft_consensus.cc:2454] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:21.805048 25083 raft_consensus.cc:2454] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:21.805375 25148 leader_election.cc:302] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:21.805539 500 raft_consensus.cc:2781] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:21.805609 500 raft_consensus.cc:482] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:21.805663 500 raft_consensus.cc:3037] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:21.806727 500 raft_consensus.cc:504] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:21.807096 500 leader_election.cc:288] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:21.807539 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df3b695402f3496f8ecf31d952ae738d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:21.807667 25083 raft_consensus.cc:3037] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:21.808621 25083 raft_consensus.cc:2454] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:21.809121 25148 leader_election.cc:302] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:21.809306 500 raft_consensus.cc:2781] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:21.809415 500 raft_consensus.cc:686] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:21.809548 500 consensus_queue.cc:237] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:21.809962 22372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df3b695402f3496f8ecf31d952ae738d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:21.810097 22372 raft_consensus.cc:3037] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:21.811136 22372 raft_consensus.cc:2454] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:21.812021 22222 catalog_manager.cc:5484] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:21.819090 22763 consensus_queue.cc:237] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:21.820374 22372 raft_consensus.cc:1261] T 038a695352ca4da58afaeb63a7785866 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:21.820608 22501 raft_consensus.cc:1261] T 038a695352ca4da58afaeb63a7785866 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:21.821121 32718 consensus_queue.cc:1035] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:21.821508 836 consensus_queue.cc:1035] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:21.823436 32718 raft_consensus.cc:2932] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:21.823861 22372 raft_consensus.cc:2932] T 038a695352ca4da58afaeb63a7785866 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:21.823976 22501 raft_consensus.cc:2932] T 038a695352ca4da58afaeb63a7785866 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:21.827600 22222 catalog_manager.cc:5484] T 038a695352ca4da58afaeb63a7785866 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index 2 to 3, VOTER 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136) evicted. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:21.830466 25193 tablet_service.cc:1514] Processing DeleteTablet for tablet 038a695352ca4da58afaeb63a7785866 with delete_type TABLET_DATA_TOMBSTONED (TS 43930dee0cd5498fa8e357a34dd71432 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:21.830685 1092 tablet_replica.cc:330] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:21.830826 1092 raft_consensus.cc:2229] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:21.830921 1092 raft_consensus.cc:2258] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:21.831254 1092 ts_tablet_manager.cc:1875] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:21.833050 1092 ts_tablet_manager.cc:1888] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:21.833122 1092 log.cc:1196] T 038a695352ca4da58afaeb63a7785866 P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/038a695352ca4da58afaeb63a7785866
I20240502 07:59:21.833734 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet 038a695352ca4da58afaeb63a7785866 (table rebalancer_extra_table_40 [id=b12bf8ee0f3a4224a3269ebae6c91866]) successfully deleted
W20240502 07:59:21.857990 22700 consensus_peers.cc:462] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:21.867723 25150 consensus_peers.cc:462] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:21.885051 25019 consensus_peers.cc:485] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:21.895032 22228 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:21.905176 22481 tablet_service.cc:1467] Processing CreateTablet for tablet 80b043e06deb4582b06d1e0e5d8a0658 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=1fc21c49c3a7460eac5995c371942c7c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:21.905766 22481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80b043e06deb4582b06d1e0e5d8a0658. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:21.907596 24931 tablet_service.cc:1467] Processing CreateTablet for tablet 7c09dfefb9c246c684884973bdc35c75 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=1fc21c49c3a7460eac5995c371942c7c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:21.907864 24931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c09dfefb9c246c684884973bdc35c75. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:21.907945 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 855b2d71ec18485dbd43acaf4fc1d986 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=1fc21c49c3a7460eac5995c371942c7c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:21.908197 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 855b2d71ec18485dbd43acaf4fc1d986. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:21.910672 24930 tablet_service.cc:1467] Processing CreateTablet for tablet 80b043e06deb4582b06d1e0e5d8a0658 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=1fc21c49c3a7460eac5995c371942c7c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:21.910946 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80b043e06deb4582b06d1e0e5d8a0658. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:21.911159 22613 tablet_service.cc:1467] Processing CreateTablet for tablet 855b2d71ec18485dbd43acaf4fc1d986 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=1fc21c49c3a7460eac5995c371942c7c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:21.911399 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 855b2d71ec18485dbd43acaf4fc1d986. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:21.913125 25193 tablet_service.cc:1467] Processing CreateTablet for tablet 7c09dfefb9c246c684884973bdc35c75 (DEFAULT_TABLE table=rebalancer_extra_table_62 [id=1fc21c49c3a7460eac5995c371942c7c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:21.913429 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c09dfefb9c246c684884973bdc35c75. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:21.915589 1120 tablet_bootstrap.cc:492] T 7c09dfefb9c246c684884973bdc35c75 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:21.916616 1120 tablet_bootstrap.cc:654] T 7c09dfefb9c246c684884973bdc35c75 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:21.916633 1189 tablet_bootstrap.cc:492] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:21.917635 1189 tablet_bootstrap.cc:654] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:21.933583 1188 tablet_bootstrap.cc:492] T 855b2d71ec18485dbd43acaf4fc1d986 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:21.934862 1120 tablet_bootstrap.cc:492] T 7c09dfefb9c246c684884973bdc35c75 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:21.934978 1120 ts_tablet_manager.cc:1367] T 7c09dfefb9c246c684884973bdc35c75 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.002s
I20240502 07:59:21.935216 1188 tablet_bootstrap.cc:654] T 855b2d71ec18485dbd43acaf4fc1d986 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:21.935434 1120 raft_consensus.cc:348] T 7c09dfefb9c246c684884973bdc35c75 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:21.935544 1120 raft_consensus.cc:374] T 7c09dfefb9c246c684884973bdc35c75 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:21.935570 1120 raft_consensus.cc:729] T 7c09dfefb9c246c684884973bdc35c75 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:21.935678 1120 consensus_queue.cc:260] T 7c09dfefb9c246c684884973bdc35c75 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:21.935916 1120 ts_tablet_manager.cc:1398] T 7c09dfefb9c246c684884973bdc35c75 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:21.937510 1189 tablet_bootstrap.cc:492] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:21.937623 1189 ts_tablet_manager.cc:1367] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 07:59:21.937975 22223 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:21.938088 1189 raft_consensus.cc:348] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:21.938395 1189 raft_consensus.cc:374] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:21.939312 1189 raft_consensus.cc:729] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:21.939532 1189 consensus_queue.cc:260] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:21.939904 1189 ts_tablet_manager.cc:1398] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:21.942045 1188 tablet_bootstrap.cc:492] T 855b2d71ec18485dbd43acaf4fc1d986 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:21.942143 1188 ts_tablet_manager.cc:1367] T 855b2d71ec18485dbd43acaf4fc1d986 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20240502 07:59:21.942638 1188 raft_consensus.cc:348] T 855b2d71ec18485dbd43acaf4fc1d986 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:21.942746 1188 raft_consensus.cc:374] T 855b2d71ec18485dbd43acaf4fc1d986 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:21.942778 1188 raft_consensus.cc:729] T 855b2d71ec18485dbd43acaf4fc1d986 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:21.942900 1188 consensus_queue.cc:260] T 855b2d71ec18485dbd43acaf4fc1d986 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:21.943204 1188 ts_tablet_manager.cc:1398] T 855b2d71ec18485dbd43acaf4fc1d986 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:21.944999 1187 tablet_bootstrap.cc:492] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:21.947414 22372 raft_consensus.cc:1261] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:21.947965 492 consensus_queue.cc:1035] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:21.949239 25084 raft_consensus.cc:1261] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:21.950510 1192 tablet_bootstrap.cc:492] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:21.951177 492 consensus_queue.cc:1035] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:21.952819 22225 catalog_manager.cc:5576] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535) does not have the latest schema for tablet df3b695402f3496f8ecf31d952ae738d (table rebalancer_dynamic_table [id=b2d5de90f1924f84b4163927d7836158]). Expected version 1 got 0
I20240502 07:59:21.955776 22230 catalog_manager.cc:5576] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733) does not have the latest schema for tablet df3b695402f3496f8ecf31d952ae738d (table rebalancer_dynamic_table [id=b2d5de90f1924f84b4163927d7836158]). Expected version 1 got 0
I20240502 07:59:21.956530 1030 tablet.cc:1722] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:21.956883 1030 tablet.cc:1620] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:21.957052 1029 tablet.cc:1722] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:21.957367 1029 tablet.cc:1620] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:21.958528 1187 tablet_bootstrap.cc:654] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:21.964601 22229 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=b2d5de90f1924f84b4163927d7836158] alter complete (version 1)
I20240502 07:59:21.965487 22221 catalog_manager.cc:5576] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535) does not have the latest schema for tablet df3b695402f3496f8ecf31d952ae738d (table rebalancer_dynamic_table [id=b2d5de90f1924f84b4163927d7836158]). Expected version 1 got 0
I20240502 07:59:21.966271 1192 tablet_bootstrap.cc:654] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:21.968842 1192 tablet_bootstrap.cc:492] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:21.968991 1192 ts_tablet_manager.cc:1367] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20240502 07:59:21.969431 1196 tablet.cc:1722] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:21.969658 1192 raft_consensus.cc:348] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:21.969846 1192 raft_consensus.cc:374] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:21.969933 1192 raft_consensus.cc:729] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:21.970211 1196 tablet.cc:1620] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:21.970990 1192 consensus_queue.cc:260] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:21.974216 1192 ts_tablet_manager.cc:1398] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20240502 07:59:21.981496 24930 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:21.982720 1187 tablet_bootstrap.cc:492] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:21.982823 1187 ts_tablet_manager.cc:1367] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.000s
I20240502 07:59:21.983297 1187 raft_consensus.cc:348] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:21.983423 1187 raft_consensus.cc:374] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:21.983493 1187 raft_consensus.cc:729] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:21.983677 1187 consensus_queue.cc:260] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:21.983971 1187 ts_tablet_manager.cc:1398] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:21.984205 1187 tablet_bootstrap.cc:492] T 80b043e06deb4582b06d1e0e5d8a0658 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:21.985261 1187 tablet_bootstrap.cc:654] T 80b043e06deb4582b06d1e0e5d8a0658 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
W20240502 07:59:21.986137 25150 consensus_peers.cc:485] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:21.986699 1187 tablet_bootstrap.cc:492] T 80b043e06deb4582b06d1e0e5d8a0658 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:21.986857 1187 ts_tablet_manager.cc:1367] T 80b043e06deb4582b06d1e0e5d8a0658 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:21.987366 1187 raft_consensus.cc:348] T 80b043e06deb4582b06d1e0e5d8a0658 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:21.987493 1187 raft_consensus.cc:374] T 80b043e06deb4582b06d1e0e5d8a0658 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:21.987548 1187 raft_consensus.cc:729] T 80b043e06deb4582b06d1e0e5d8a0658 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
W20240502 07:59:21.987672 25150 consensus_peers.cc:485] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:21.987725 1187 consensus_queue.cc:260] T 80b043e06deb4582b06d1e0e5d8a0658 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:21.988024 1187 ts_tablet_manager.cc:1398] T 80b043e06deb4582b06d1e0e5d8a0658 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:22.008136 24952 raft_consensus.cc:1261] T 5034380b5c0846faa1494140d67203cc P 1aecef81733c47b3b468504934357231 [term 2 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 07:59:22.008697 365 consensus_queue.cc:1035] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:22.011911 22745 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:22.016985 25063 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:22.021800 22481 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 07:59:22.023963 25193 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:22.026479 22351 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:22.026983 22613 tablet_service.cc:1429] Tablet server has 24 leaders and 0 scanners
I20240502 07:59:22.028015 32485 raft_consensus.cc:482] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:22.028110 32485 raft_consensus.cc:504] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:22.028475 32485 leader_election.cc:288] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:22.029021 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c09dfefb9c246c684884973bdc35c75" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:22.029155 25215 raft_consensus.cc:2454] T 7c09dfefb9c246c684884973bdc35c75 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
W20240502 07:59:22.029525 24887 leader_election.cc:334] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:22.029604 24888 leader_election.cc:302] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters: 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:22.029863 32485 raft_consensus.cc:2781] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:22.029912 32485 raft_consensus.cc:482] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:22.029945 32485 raft_consensus.cc:3037] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:22.030989 32485 raft_consensus.cc:504] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:22.031332 32485 leader_election.cc:288] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:22.031896 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c09dfefb9c246c684884973bdc35c75" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:22.032006 25215 raft_consensus.cc:3037] T 7c09dfefb9c246c684884973bdc35c75 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
W20240502 07:59:22.032212 22438 consensus_peers.cc:485] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:22.032655 24887 leader_election.cc:334] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:22.033067 25215 raft_consensus.cc:2454] T 7c09dfefb9c246c684884973bdc35c75 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:22.034188 24888 leader_election.cc:302] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters: 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:22.037400 1024 raft_consensus.cc:482] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:22.037509 1024 raft_consensus.cc:504] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:22.037917 1024 leader_election.cc:288] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:22.039134 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855b2d71ec18485dbd43acaf4fc1d986" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:22.039280 22763 raft_consensus.cc:2454] T 855b2d71ec18485dbd43acaf4fc1d986 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:22.039637 22570 leader_election.cc:302] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
W20240502 07:59:22.039836 22569 leader_election.cc:334] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:22.039857 1024 raft_consensus.cc:2781] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:22.040256 1024 raft_consensus.cc:482] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:22.040311 1024 raft_consensus.cc:3037] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:22.041239 1024 raft_consensus.cc:504] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:22.041560 1024 leader_election.cc:288] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:22.043406 601 raft_consensus.cc:2781] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:22.043612 601 raft_consensus.cc:686] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:22.043624 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855b2d71ec18485dbd43acaf4fc1d986" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:22.043735 22763 raft_consensus.cc:3037] T 855b2d71ec18485dbd43acaf4fc1d986 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:22.043757 601 consensus_queue.cc:237] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:22.044873 22763 raft_consensus.cc:2454] T 855b2d71ec18485dbd43acaf4fc1d986 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:22.045433 22570 leader_election.cc:302] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:22.045717 1024 raft_consensus.cc:2781] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:22.045792 1024 raft_consensus.cc:686] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:59:22.045899 1024 consensus_queue.cc:237] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
W20240502 07:59:22.046993 22569 leader_election.cc:334] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:22.047039 22221 catalog_manager.cc:5484] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:22.048313 22499 raft_consensus.cc:1261] T 3d1511fc27c24820afaa6e41161be8bb P fabddb16c85944648a710ec52fc7244c [term 2 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 07:59:22.048338 22224 catalog_manager.cc:5484] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:22.049584 500 consensus_queue.cc:1035] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 07:59:22.057083 25019 consensus_peers.cc:485] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:22.066123 25019 consensus_peers.cc:485] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:22.067148 22221 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:22.071789 22634 raft_consensus.cc:1261] T 10ae14d069e048ccba7fdfc79951cec0 P e51b10af49484aaa8f0267aebc1bf27f [term 2 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 07:59:22.072192 308 consensus_queue.cc:1035] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 07:59:22.073110 22307 consensus_peers.cc:485] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:22.077165 1030 tablet.cc:1722] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:22.077525 1030 tablet.cc:1620] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432: MemRowSet was empty: no flush needed.
I20240502 07:59:22.077877 1029 tablet.cc:1722] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:22.078173 1029 tablet.cc:1620] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:22.084189 22229 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=b2d5de90f1924f84b4163927d7836158] alter complete (version 2)
I20240502 07:59:22.086249 24952 raft_consensus.cc:1261] T 751f8c9cbf684b01b90e7bd26685f8bf P 1aecef81733c47b3b468504934357231 [term 2 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 07:59:22.086671 1196 tablet.cc:1722] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:22.086870 365 consensus_queue.cc:1035] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:22.087006 1196 tablet.cc:1620] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
W20240502 07:59:22.092067 22216 catalog_manager.cc:4573] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): CreateTablet RPC failed for tablet 7c09dfefb9c246c684884973bdc35c75: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:22.099367 25019 consensus_peers.cc:485] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:22.101117 22763 raft_consensus.cc:1261] T 5034380b5c0846faa1494140d67203cc P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 07:59:22.101647 32660 consensus_queue.cc:1035] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:22.104482 24952 raft_consensus.cc:1261] T 8fcb3fef170343d494d8dfbba33052d9 P 1aecef81733c47b3b468504934357231 [term 2 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 07:59:22.105207 1021 consensus_queue.cc:1035] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 07:59:22.117270 25019 consensus_peers.cc:485] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:22.132378 25084 raft_consensus.cc:1261] T 712a88cd545745e8937d295266e85f04 P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Refusing update from remote peer 43930dee0cd5498fa8e357a34dd71432: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 07:59:22.133198 492 consensus_queue.cc:1035] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 07:59:22.141922 22700 consensus_peers.cc:485] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:22.143780 22764 consensus_queue.cc:237] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } }
I20240502 07:59:22.144899 25215 raft_consensus.cc:1261] T 5242aa2d2d9c477994026edb0d42051c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:22.144997 22634 raft_consensus.cc:1261] T 5242aa2d2d9c477994026edb0d42051c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:22.145417 32718 consensus_queue.cc:1035] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:22.145527 22764 raft_consensus.cc:1261] T bf443ada59854e6da532aa5832a9a564 P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 07:59:22.145598 31905 consensus_queue.cc:1035] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:22.145818 32660 consensus_queue.cc:1035] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:22.146358 666 raft_consensus.cc:482] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:22.146457 666 raft_consensus.cc:504] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:22.146927 666 leader_election.cc:288] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:22.147203 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80b043e06deb4582b06d1e0e5d8a0658" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:22.147354 24951 raft_consensus.cc:2454] T 80b043e06deb4582b06d1e0e5d8a0658 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:22.147874 32718 raft_consensus.cc:2932] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:22.148042 22439 leader_election.cc:302] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:22.148346 666 raft_consensus.cc:2781] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:22.148401 666 raft_consensus.cc:482] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:22.148440 666 raft_consensus.cc:3037] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
W20240502 07:59:22.149047 22438 leader_election.cc:334] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:22.150417 25215 raft_consensus.cc:2932] T 5242aa2d2d9c477994026edb0d42051c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:22.150401 22634 raft_consensus.cc:2932] T 5242aa2d2d9c477994026edb0d42051c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:22.152305 666 raft_consensus.cc:504] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:22.152663 666 leader_election.cc:288] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661)
I20240502 07:59:22.152793 22230 catalog_manager.cc:5484] T 5242aa2d2d9c477994026edb0d42051c P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } }
I20240502 07:59:22.153276 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80b043e06deb4582b06d1e0e5d8a0658" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:22.153432 24951 raft_consensus.cc:3037] T 80b043e06deb4582b06d1e0e5d8a0658 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:22.154511 24951 raft_consensus.cc:2454] T 80b043e06deb4582b06d1e0e5d8a0658 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
W20240502 07:59:22.155283 22438 leader_election.cc:334] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:22.155603 22439 leader_election.cc:302] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1aecef81733c47b3b468504934357231, fabddb16c85944648a710ec52fc7244c; no voters: 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:22.156064 666 raft_consensus.cc:2781] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:22.156229 666 raft_consensus.cc:686] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:59:22.156394 666 consensus_queue.cc:237] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:22.158023 24930 tablet_service.cc:1514] Processing DeleteTablet for tablet 5242aa2d2d9c477994026edb0d42051c with delete_type TABLET_DATA_TOMBSTONED (TS 1aecef81733c47b3b468504934357231 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:22.158838 22230 catalog_manager.cc:5484] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
W20240502 07:59:22.162475 25019 consensus_peers.cc:485] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:22.162918 1247 tablet_replica.cc:330] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:22.163066 1247 raft_consensus.cc:2229] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:22.163214 1247 raft_consensus.cc:2258] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:22.163600 1247 ts_tablet_manager.cc:1875] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:22.165412 1247 ts_tablet_manager.cc:1888] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:22.165477 1247 log.cc:1196] T 5242aa2d2d9c477994026edb0d42051c P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/5242aa2d2d9c477994026edb0d42051c
I20240502 07:59:22.166014 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet 5242aa2d2d9c477994026edb0d42051c (table rebalancer_extra_table_12 [id=97589e13ee074648b68057aa77a8d10d]) successfully deleted
I20240502 07:59:22.166759 25215 raft_consensus.cc:1261] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 43930dee0cd5498fa8e357a34dd71432 [term 2 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 07:59:22.167202 32660 consensus_queue.cc:1035] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:22.179868 25215 raft_consensus.cc:1261] T 79a67cfa3c554f2d85a520bb8f5ccc81 P 43930dee0cd5498fa8e357a34dd71432 [term 2 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 07:59:22.180275 1245 consensus_queue.cc:1035] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 07:59:22.188020 22569 consensus_peers.cc:485] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:22.195200 22230 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:22.196049 22372 raft_consensus.cc:1261] T 01d9a73be1fd472db39860e538b9c97e P b533a7d8d97943609c6e28e3e36b78a4 [term 2 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 07:59:22.196468 32718 consensus_queue.cc:1035] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:22.199148 22370 raft_consensus.cc:1261] T 5ca68d8eec874332b74df02eb55e0ff9 P b533a7d8d97943609c6e28e3e36b78a4 [term 2 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 07:59:22.199757 1114 consensus_queue.cc:1035] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20240502 07:59:22.201229 1030 tablet.cc:1722] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:22.202668 1196 tablet.cc:1722] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:22.203040 1029 tablet.cc:1722] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:22.209568 22230 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=b2d5de90f1924f84b4163927d7836158] alter complete (version 3)
I20240502 07:59:22.225761 25215 raft_consensus.cc:1261] T b2abf00f982e450fbe0124226789324a P 43930dee0cd5498fa8e357a34dd71432 [term 2 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 07:59:22.227326 308 consensus_queue.cc:1035] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 07:59:22.236052 22307 consensus_peers.cc:485] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:22.244153 22764 raft_consensus.cc:1261] T 9f916726456f4aa8936a9366468bc471 P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20240502 07:59:22.244583 32660 consensus_queue.cc:1035] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20240502 07:59:22.265396 22569 consensus_peers.cc:462] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:22.273734 25150 consensus_peers.cc:462] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:22.314026 22228 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:22.314180 22228 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
W20240502 07:59:22.319037 22700 consensus_peers.cc:462] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:22.321204 22228 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet df3b695402f3496f8ecf31d952ae738d
I20240502 07:59:22.322026 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet df3b695402f3496f8ecf31d952ae738d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:22 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:22.322269 25193 tablet_service.cc:1514] Processing DeleteTablet for tablet df3b695402f3496f8ecf31d952ae738d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:22 UTC) from {username='slave'} at 127.0.0.1:46972
I20240502 07:59:22.322453 1092 tablet_replica.cc:330] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432: stopping tablet replica
I20240502 07:59:22.322590 1092 raft_consensus.cc:2229] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:22.322795 1092 raft_consensus.cc:2258] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:22.323463 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet df3b695402f3496f8ecf31d952ae738d with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:22 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:22.323503 1092 ts_tablet_manager.cc:1875] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:22.326537 1319 tablet_replica.cc:330] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:22.327091 1092 ts_tablet_manager.cc:1888] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:22.326925 1319 raft_consensus.cc:2229] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:22.327224 1092 log.cc:1196] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/wals/df3b695402f3496f8ecf31d952ae738d
I20240502 07:59:22.327348 1319 raft_consensus.cc:2258] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:22.327934 1319 ts_tablet_manager.cc:1875] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:22.328164 1318 tablet_replica.cc:330] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:22.328398 1092 ts_tablet_manager.cc:1909] T df3b695402f3496f8ecf31d952ae738d P 43930dee0cd5498fa8e357a34dd71432: Deleting consensus metadata
I20240502 07:59:22.329953 1318 raft_consensus.cc:2229] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:22.330781 1318 raft_consensus.cc:2258] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:22.330825 1319 ts_tablet_manager.cc:1888] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:22.330895 22217 catalog_manager.cc:4830] TS 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591): tablet df3b695402f3496f8ecf31d952ae738d (table rebalancer_dynamic_table_new_name [id=b2d5de90f1924f84b4163927d7836158]) successfully deleted
I20240502 07:59:22.330957 1319 log.cc:1196] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/df3b695402f3496f8ecf31d952ae738d
I20240502 07:59:22.331343 1319 ts_tablet_manager.cc:1909] T df3b695402f3496f8ecf31d952ae738d P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:59:22.331493 1318 ts_tablet_manager.cc:1875] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:22.331833 25063 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:22.334702 1318 ts_tablet_manager.cc:1888] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:22.334825 1318 log.cc:1196] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/df3b695402f3496f8ecf31d952ae738d
I20240502 07:59:22.335290 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet df3b695402f3496f8ecf31d952ae738d (table rebalancer_dynamic_table_new_name [id=b2d5de90f1924f84b4163927d7836158]) successfully deleted
I20240502 07:59:22.335197 1318 ts_tablet_manager.cc:1909] T df3b695402f3496f8ecf31d952ae738d P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:59:22.336634 22351 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:22.337167 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet df3b695402f3496f8ecf31d952ae738d (table rebalancer_dynamic_table_new_name [id=b2d5de90f1924f84b4163927d7836158]) successfully deleted
I20240502 07:59:22.339283 24930 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:22.343799 22481 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:22.343948 22613 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240502 07:59:22.344434 22745 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:22.345847 25193 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:22.423168 22634 consensus_queue.cc:237] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:22.424072 22228 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:22.424770 25084 raft_consensus.cc:1261] T 2241a741bf85433b9fea512e9c530416 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:22.426020 1024 consensus_queue.cc:1035] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:22.426792 25215 raft_consensus.cc:1261] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:22.427461 1024 consensus_queue.cc:1035] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:22.429868 1114 raft_consensus.cc:2932] T 2241a741bf85433b9fea512e9c530416 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:22.430297 25084 raft_consensus.cc:2932] T 2241a741bf85433b9fea512e9c530416 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:22.431269 25215 raft_consensus.cc:2932] T 2241a741bf85433b9fea512e9c530416 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:22.431785 22613 tablet_service.cc:1467] Processing CreateTablet for tablet 8ef453d0a2eb4873b25903242ca39a96 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=1e42a48389cd4b5cb538489ed972801a]), partition=
I20240502 07:59:22.431906 22481 tablet_service.cc:1467] Processing CreateTablet for tablet 8ef453d0a2eb4873b25903242ca39a96 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=1e42a48389cd4b5cb538489ed972801a]), partition=
I20240502 07:59:22.431985 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ef453d0a2eb4873b25903242ca39a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:22.432158 22481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ef453d0a2eb4873b25903242ca39a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:22.434409 1189 tablet_bootstrap.cc:492] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:22.435191 22228 catalog_manager.cc:5484] T 2241a741bf85433b9fea512e9c530416 P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: config changed from index 6 to 7, VOTER f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132) evicted. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:22.435340 1189 tablet_bootstrap.cc:654] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:22.436316 1192 tablet_bootstrap.cc:492] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:22.437435 1192 tablet_bootstrap.cc:654] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:22.439216 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet 2241a741bf85433b9fea512e9c530416 with delete_type TABLET_DATA_TOMBSTONED (TS f00f3a1876444ad2bf9c919068ab8014 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:22.440965 1340 tablet_replica.cc:330] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:22.441255 1340 raft_consensus.cc:2229] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:22.441532 1340 raft_consensus.cc:2258] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:22.442291 1340 ts_tablet_manager.cc:1875] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:22.443982 1192 tablet_bootstrap.cc:492] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:22.444074 1192 ts_tablet_manager.cc:1367] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20240502 07:59:22.444605 1192 raft_consensus.cc:348] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:22.444727 1192 raft_consensus.cc:374] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:22.444758 1192 raft_consensus.cc:729] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:22.444880 1192 consensus_queue.cc:260] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:22.445173 1192 ts_tablet_manager.cc:1398] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:22.445487 1340 ts_tablet_manager.cc:1888] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:22.445583 1340 log.cc:1196] T 2241a741bf85433b9fea512e9c530416 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/2241a741bf85433b9fea512e9c530416
I20240502 07:59:22.447077 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet 2241a741bf85433b9fea512e9c530416 (table rebalance_test_table_11 [id=423ebbc6961345159cc3d4c9668fe1dd]) successfully deleted
I20240502 07:59:22.447441 1189 tablet_bootstrap.cc:492] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:22.447540 1189 ts_tablet_manager.cc:1367] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.001s
I20240502 07:59:22.447961 1189 raft_consensus.cc:348] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:22.448057 1189 raft_consensus.cc:374] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:22.448092 1189 raft_consensus.cc:729] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:22.448221 1189 consensus_queue.cc:260] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:22.448469 1189 ts_tablet_manager.cc:1398] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
W20240502 07:59:22.458063 24887 consensus_peers.cc:485] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:22.483155 22569 consensus_peers.cc:485] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:22.489241 1212 consensus_queue.cc:1035] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 07:59:22.519917 22438 consensus_peers.cc:485] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:22.557264 22569 consensus_peers.cc:485] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:22.561826 25150 consensus_peers.cc:485] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:22.581527 1245 raft_consensus.cc:482] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:22.581682 1245 raft_consensus.cc:504] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:22.582037 1245 leader_election.cc:288] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:22.582469 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ef453d0a2eb4873b25903242ca39a96" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:22.582626 22634 raft_consensus.cc:2454] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
W20240502 07:59:22.583039 22438 leader_election.cc:334] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:22.583292 22439 leader_election.cc:302] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters: 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:22.583518 1245 raft_consensus.cc:2781] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:22.583572 1245 raft_consensus.cc:482] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:22.583607 1245 raft_consensus.cc:3037] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:22.584820 1245 raft_consensus.cc:504] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:22.585186 1245 leader_election.cc:288] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:22.585997 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ef453d0a2eb4873b25903242ca39a96" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
W20240502 07:59:22.586050 22438 leader_election.cc:334] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:22.586160 22634 raft_consensus.cc:3037] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:22.587272 22634 raft_consensus.cc:2454] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:22.588033 22439 leader_election.cc:302] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters: 4b4e7eeb7f3f42119c0363a79febc54b
I20240502 07:59:22.588898 1245 raft_consensus.cc:2781] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:22.589035 1245 raft_consensus.cc:686] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:59:22.589226 1245 consensus_queue.cc:237] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:22.591526 22225 catalog_manager.cc:5484] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:22.598665 1114 consensus_queue.cc:1035] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 07:59:22.606416 24887 consensus_peers.cc:485] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:22.621264 25193 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:22.623785 22351 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:22.624692 25063 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:22.626735 1257 consensus_queue.cc:1035] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:22.627681 22481 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:22.632658 24930 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:22.637511 22745 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
W20240502 07:59:22.637882 22569 consensus_peers.cc:485] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:22.647529 22613 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
W20240502 07:59:22.676466 22438 consensus_peers.cc:485] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:22.677937 24887 consensus_peers.cc:485] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:22.684676 22225 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:22.695955 22481 tablet_service.cc:1467] Processing CreateTablet for tablet 6d605515417c44078b96012144f4110d (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=97100f6dec3647c2a9088b37b67b56c3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:22.696265 22481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d605515417c44078b96012144f4110d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:22.697567 22482 tablet_service.cc:1467] Processing CreateTablet for tablet c08b5f0114744488a3065bfc4d04d993 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=97100f6dec3647c2a9088b37b67b56c3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:22.697835 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c08b5f0114744488a3065bfc4d04d993. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:22.698669 1192 tablet_bootstrap.cc:492] T 6d605515417c44078b96012144f4110d P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:22.699573 1192 tablet_bootstrap.cc:654] T 6d605515417c44078b96012144f4110d P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:22.699843 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 10cdb9745afd4a67b68f8e4e4e14f81b (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=97100f6dec3647c2a9088b37b67b56c3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:22.700045 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10cdb9745afd4a67b68f8e4e4e14f81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:22.700541 22225 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:22.701759 1192 tablet_bootstrap.cc:492] T 6d605515417c44078b96012144f4110d P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:22.701843 1192 ts_tablet_manager.cc:1367] T 6d605515417c44078b96012144f4110d P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 07:59:22.702315 1192 raft_consensus.cc:348] T 6d605515417c44078b96012144f4110d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:22.702442 1192 raft_consensus.cc:374] T 6d605515417c44078b96012144f4110d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:22.702468 1192 raft_consensus.cc:729] T 6d605515417c44078b96012144f4110d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:22.702580 1192 consensus_queue.cc:260] T 6d605515417c44078b96012144f4110d P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:22.702858 1192 ts_tablet_manager.cc:1398] T 6d605515417c44078b96012144f4110d P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 07:59:22.703015 25194 tablet_service.cc:1467] Processing CreateTablet for tablet 10cdb9745afd4a67b68f8e4e4e14f81b (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=97100f6dec3647c2a9088b37b67b56c3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:22.703357 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10cdb9745afd4a67b68f8e4e4e14f81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:22.703387 25193 tablet_service.cc:1467] Processing CreateTablet for tablet 6d605515417c44078b96012144f4110d (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=97100f6dec3647c2a9088b37b67b56c3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:22.703608 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d605515417c44078b96012144f4110d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:22.704653 22745 tablet_service.cc:1467] Processing CreateTablet for tablet c08b5f0114744488a3065bfc4d04d993 (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=97100f6dec3647c2a9088b37b67b56c3]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:22.704830 22743 tablet_service.cc:1467] Processing CreateTablet for tablet 10cdb9745afd4a67b68f8e4e4e14f81b (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=97100f6dec3647c2a9088b37b67b56c3]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:22.704968 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c08b5f0114744488a3065bfc4d04d993. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:22.705129 22743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10cdb9745afd4a67b68f8e4e4e14f81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:22.707649 1192 tablet_bootstrap.cc:492] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:22.708600 1192 tablet_bootstrap.cc:654] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:22.711977 22351 tablet_service.cc:1467] Processing CreateTablet for tablet 6d605515417c44078b96012144f4110d (DEFAULT_TABLE table=rebalancer_extra_table_64 [id=97100f6dec3647c2a9088b37b67b56c3]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:22.712239 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d605515417c44078b96012144f4110d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:22.719110 1407 tablet_bootstrap.cc:492] T 6d605515417c44078b96012144f4110d P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:22.720530 1407 tablet_bootstrap.cc:654] T 6d605515417c44078b96012144f4110d P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:22.721212 365 raft_consensus.cc:482] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:22.721498 1404 tablet_bootstrap.cc:492] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:22.721311 365 raft_consensus.cc:504] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:22.721937 365 leader_election.cc:288] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:22.722438 1403 tablet_bootstrap.cc:492] T 10cdb9745afd4a67b68f8e4e4e14f81b P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:22.723361 1403 tablet_bootstrap.cc:654] T 10cdb9745afd4a67b68f8e4e4e14f81b P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:22.723830 1404 tablet_bootstrap.cc:654] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
W20240502 07:59:22.724710 25020 leader_election.cc:341] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): Not found: Tablet not found: 05911d56b0344eb8a53aba5f4bf0170c
I20240502 07:59:22.725688 1405 tablet_bootstrap.cc:492] T 10cdb9745afd4a67b68f8e4e4e14f81b P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:22.727056 1405 tablet_bootstrap.cc:654] T 10cdb9745afd4a67b68f8e4e4e14f81b P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:22.728096 1192 tablet_bootstrap.cc:492] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:22.728186 1192 ts_tablet_manager.cc:1367] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.001s
I20240502 07:59:22.728356 1404 tablet_bootstrap.cc:492] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:22.728497 1404 ts_tablet_manager.cc:1367] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:22.728608 1192 raft_consensus.cc:348] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:22.728705 1192 raft_consensus.cc:374] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:22.728732 1192 raft_consensus.cc:729] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:22.728854 1192 consensus_queue.cc:260] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:22.728966 1404 raft_consensus.cc:348] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:22.729061 1404 raft_consensus.cc:374] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:22.729090 1404 raft_consensus.cc:729] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:22.729127 1192 ts_tablet_manager.cc:1398] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:22.729210 1404 consensus_queue.cc:260] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:22.729557 1403 tablet_bootstrap.cc:492] T 10cdb9745afd4a67b68f8e4e4e14f81b P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:22.729665 1403 ts_tablet_manager.cc:1367] T 10cdb9745afd4a67b68f8e4e4e14f81b P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
W20240502 07:59:22.730033 25019 leader_election.cc:334] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:22.730113 25019 leader_election.cc:302] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055; no voters: 4b4e7eeb7f3f42119c0363a79febc54b, b533a7d8d97943609c6e28e3e36b78a4
I20240502 07:59:22.730324 365 raft_consensus.cc:2726] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240502 07:59:22.730824 1404 ts_tablet_manager.cc:1398] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:22.730993 1404 tablet_bootstrap.cc:492] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:22.732082 1403 raft_consensus.cc:348] T 10cdb9745afd4a67b68f8e4e4e14f81b P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:22.732272 1403 raft_consensus.cc:374] T 10cdb9745afd4a67b68f8e4e4e14f81b P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:22.732358 1403 raft_consensus.cc:729] T 10cdb9745afd4a67b68f8e4e4e14f81b P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:22.732535 1403 consensus_queue.cc:260] T 10cdb9745afd4a67b68f8e4e4e14f81b P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:22.732640 1404 tablet_bootstrap.cc:654] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:22.735533 1403 ts_tablet_manager.cc:1398] T 10cdb9745afd4a67b68f8e4e4e14f81b P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
I20240502 07:59:22.736415 1405 tablet_bootstrap.cc:492] T 10cdb9745afd4a67b68f8e4e4e14f81b P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:22.736549 1405 ts_tablet_manager.cc:1367] T 10cdb9745afd4a67b68f8e4e4e14f81b P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 07:59:22.736689 1407 tablet_bootstrap.cc:492] T 6d605515417c44078b96012144f4110d P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:22.736828 1407 ts_tablet_manager.cc:1367] T 6d605515417c44078b96012144f4110d P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 07:59:22.736840 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05911d56b0344eb8a53aba5f4bf0170c" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:22.736970 25215 raft_consensus.cc:2454] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:22.738693 1407 raft_consensus.cc:348] T 6d605515417c44078b96012144f4110d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:22.738870 1407 raft_consensus.cc:374] T 6d605515417c44078b96012144f4110d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:22.738955 1407 raft_consensus.cc:729] T 6d605515417c44078b96012144f4110d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
W20240502 07:59:22.739729 22569 consensus_peers.cc:485] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:22.739884 1405 raft_consensus.cc:348] T 10cdb9745afd4a67b68f8e4e4e14f81b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:22.740006 1405 raft_consensus.cc:374] T 10cdb9745afd4a67b68f8e4e4e14f81b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:22.740037 1405 raft_consensus.cc:729] T 10cdb9745afd4a67b68f8e4e4e14f81b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:22.740149 1405 consensus_queue.cc:260] T 10cdb9745afd4a67b68f8e4e4e14f81b P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:22.740300 1407 consensus_queue.cc:260] T 6d605515417c44078b96012144f4110d P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:22.740512 1405 ts_tablet_manager.cc:1398] T 10cdb9745afd4a67b68f8e4e4e14f81b P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:22.740648 1407 ts_tablet_manager.cc:1398] T 6d605515417c44078b96012144f4110d P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:22.740723 1405 tablet_bootstrap.cc:492] T c08b5f0114744488a3065bfc4d04d993 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:22.741899 1405 tablet_bootstrap.cc:654] T c08b5f0114744488a3065bfc4d04d993 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:22.745662 22634 raft_consensus.cc:1261] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer fabddb16c85944648a710ec52fc7244c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:22.746064 666 consensus_queue.cc:1035] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 07:59:22.747138 22307 consensus_peers.cc:485] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:22.748973 22438 consensus_peers.cc:485] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:22.750435 22225 catalog_manager.cc:5576] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703) does not have the latest schema for tablet 8ef453d0a2eb4873b25903242ca39a96 (table rebalancer_dynamic_table [id=1e42a48389cd4b5cb538489ed972801a]). Expected version 1 got 0
I20240502 07:59:22.754402 1405 tablet_bootstrap.cc:492] T c08b5f0114744488a3065bfc4d04d993 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:22.754563 1405 ts_tablet_manager.cc:1367] T c08b5f0114744488a3065bfc4d04d993 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 07:59:22.755126 1405 raft_consensus.cc:348] T c08b5f0114744488a3065bfc4d04d993 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:22.755302 1405 raft_consensus.cc:374] T c08b5f0114744488a3065bfc4d04d993 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:22.755398 1405 raft_consensus.cc:729] T c08b5f0114744488a3065bfc4d04d993 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:22.755576 1405 consensus_queue.cc:260] T c08b5f0114744488a3065bfc4d04d993 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:22.755986 1405 ts_tablet_manager.cc:1398] T c08b5f0114744488a3065bfc4d04d993 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240502 07:59:22.756836 24887 consensus_peers.cc:485] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:22.765250 24887 consensus_peers.cc:485] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:22.768128 1421 tablet.cc:1722] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:22.768493 1421 tablet.cc:1620] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c: MemRowSet was empty: no flush needed.
I20240502 07:59:22.770915 22224 catalog_manager.cc:5576] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703) does not have the latest schema for tablet 8ef453d0a2eb4873b25903242ca39a96 (table rebalancer_dynamic_table [id=1e42a48389cd4b5cb538489ed972801a]). Expected version 1 got 0
I20240502 07:59:22.771809 1404 tablet_bootstrap.cc:492] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:22.771914 1404 ts_tablet_manager.cc:1367] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.000s
I20240502 07:59:22.772403 1404 raft_consensus.cc:348] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:22.772523 1404 raft_consensus.cc:374] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:22.772557 1404 raft_consensus.cc:729] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:22.772678 1404 consensus_queue.cc:260] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:22.772984 1404 ts_tablet_manager.cc:1398] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:22.777272 22224 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=1e42a48389cd4b5cb538489ed972801a] alter complete (version 1)
W20240502 07:59:22.778295 25150 consensus_peers.cc:485] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:22.781927 1422 tablet.cc:1722] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:22.782286 1422 tablet.cc:1620] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
W20240502 07:59:22.797569 25150 consensus_peers.cc:485] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:22.799492 22438 consensus_peers.cc:485] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:22.835507 492 raft_consensus.cc:482] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:22.835633 492 raft_consensus.cc:504] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:22.836019 492 leader_election.cc:288] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:22.836390 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10cdb9745afd4a67b68f8e4e4e14f81b" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:22.836535 25084 raft_consensus.cc:2454] T 10cdb9745afd4a67b68f8e4e4e14f81b P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:22.836505 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10cdb9745afd4a67b68f8e4e4e14f81b" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:22.836627 22764 raft_consensus.cc:2454] T 10cdb9745afd4a67b68f8e4e4e14f81b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:22.836831 25148 leader_election.cc:302] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:22.837003 492 raft_consensus.cc:2781] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:22.837065 492 raft_consensus.cc:482] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:22.837118 492 raft_consensus.cc:3037] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:22.838266 492 raft_consensus.cc:504] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:22.838622 492 leader_election.cc:288] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:22.838913 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10cdb9745afd4a67b68f8e4e4e14f81b" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:22.839023 25084 raft_consensus.cc:3037] T 10cdb9745afd4a67b68f8e4e4e14f81b P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:22.839025 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10cdb9745afd4a67b68f8e4e4e14f81b" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:22.839141 22764 raft_consensus.cc:3037] T 10cdb9745afd4a67b68f8e4e4e14f81b P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:22.840034 25084 raft_consensus.cc:2454] T 10cdb9745afd4a67b68f8e4e4e14f81b P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:22.840034 22764 raft_consensus.cc:2454] T 10cdb9745afd4a67b68f8e4e4e14f81b P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:22.840318 25151 leader_election.cc:302] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:22.841147 492 raft_consensus.cc:2781] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:22.841238 492 raft_consensus.cc:686] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:22.841398 492 consensus_queue.cc:237] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
W20240502 07:59:22.842448 25019 consensus_peers.cc:485] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:22.842659 22569 consensus_peers.cc:462] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:22.846240 22229 catalog_manager.cc:5484] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:22.851411 666 raft_consensus.cc:482] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:22.851518 666 raft_consensus.cc:504] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:22.851892 666 leader_election.cc:288] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
W20240502 07:59:22.852072 25150 consensus_peers.cc:485] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:22.853274 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c08b5f0114744488a3065bfc4d04d993" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:22.853423 22764 raft_consensus.cc:2454] T c08b5f0114744488a3065bfc4d04d993 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:22.853724 22439 leader_election.cc:302] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
W20240502 07:59:22.854045 22438 leader_election.cc:334] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:22.854910 666 raft_consensus.cc:2781] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:22.854967 666 raft_consensus.cc:482] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:22.855005 666 raft_consensus.cc:3037] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:22.856084 666 raft_consensus.cc:504] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:22.856433 666 leader_election.cc:288] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:22.856849 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c08b5f0114744488a3065bfc4d04d993" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:22.856977 22764 raft_consensus.cc:3037] T c08b5f0114744488a3065bfc4d04d993 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:22.858112 22764 raft_consensus.cc:2454] T c08b5f0114744488a3065bfc4d04d993 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:22.858566 22439 leader_election.cc:302] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f00f3a1876444ad2bf9c919068ab8014, fabddb16c85944648a710ec52fc7244c; no voters:
W20240502 07:59:22.858875 25150 consensus_peers.cc:462] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:22.859921 22438 leader_election.cc:334] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:22.859953 666 raft_consensus.cc:2781] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:22.860033 666 raft_consensus.cc:686] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:59:22.860162 666 consensus_queue.cc:237] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:22.862911 22222 catalog_manager.cc:5484] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:22.882535 22229 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
W20240502 07:59:22.887166 25019 consensus_peers.cc:485] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:22.887579 22569 consensus_peers.cc:485] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:22.889045 1421 tablet.cc:1722] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:22.889386 1421 tablet.cc:1620] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c: MemRowSet was empty: no flush needed.
I20240502 07:59:22.890023 1422 tablet.cc:1722] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:22.890323 1422 tablet.cc:1620] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
I20240502 07:59:22.895239 22229 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=1e42a48389cd4b5cb538489ed972801a] alter complete (version 2)
W20240502 07:59:22.917088 25019 consensus_peers.cc:485] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:22.931713 22700 consensus_peers.cc:462] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:22.950354 22438 consensus_peers.cc:485] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:22.993378 22307 consensus_peers.cc:485] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:22.996763 22229 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:23.002390 1421 tablet.cc:1722] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:23.003254 1422 tablet.cc:1722] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:23.007747 22229 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=1e42a48389cd4b5cb538489ed972801a] alter complete (version 3)
I20240502 07:59:23.007834 492 raft_consensus.cc:482] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:23.007969 492 raft_consensus.cc:504] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:23.008663 492 leader_election.cc:288] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:23.008915 22499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d605515417c44078b96012144f4110d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:23.009042 22499 raft_consensus.cc:2454] T 6d605515417c44078b96012144f4110d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:23.009413 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d605515417c44078b96012144f4110d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:23.009549 22371 raft_consensus.cc:2454] T 6d605515417c44078b96012144f4110d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:23.010061 25148 leader_election.cc:302] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:23.010212 492 raft_consensus.cc:2781] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:23.010257 492 raft_consensus.cc:482] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:23.010308 492 raft_consensus.cc:3037] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:23.011636 492 raft_consensus.cc:504] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:23.012207 492 leader_election.cc:288] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:23.012269 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d605515417c44078b96012144f4110d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:23.012372 22371 raft_consensus.cc:3037] T 6d605515417c44078b96012144f4110d P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:23.012445 22499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d605515417c44078b96012144f4110d" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:23.012564 22499 raft_consensus.cc:3037] T 6d605515417c44078b96012144f4110d P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:23.013291 22371 raft_consensus.cc:2454] T 6d605515417c44078b96012144f4110d P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:23.013443 22499 raft_consensus.cc:2454] T 6d605515417c44078b96012144f4110d P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:23.013623 25151 leader_election.cc:302] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:23.013788 492 raft_consensus.cc:2781] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:23.014065 492 raft_consensus.cc:686] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:23.014215 492 consensus_queue.cc:237] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
W20240502 07:59:23.015630 22700 consensus_peers.cc:485] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:23.016532 22223 catalog_manager.cc:5484] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
W20240502 07:59:23.027204 22307 consensus_peers.cc:485] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:23.036142 22307 consensus_peers.cc:485] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:23.052690 24930 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:23.052768 25063 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:23.052964 25194 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:23.054519 22351 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:23.057823 22745 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
W20240502 07:59:23.059628 22438 consensus_peers.cc:485] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:23.060369 22482 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:23.070345 22613 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
W20240502 07:59:23.094800 22216 catalog_manager.cc:4573] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): CreateTablet RPC failed for tablet 855b2d71ec18485dbd43acaf4fc1d986: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:23.111469 22223 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:23.111627 22223 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:23.114229 22223 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet 8ef453d0a2eb4873b25903242ca39a96
I20240502 07:59:23.114893 22482 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ef453d0a2eb4873b25903242ca39a96 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:23 UTC) from {username='slave'} at 127.0.0.1:44562
I20240502 07:59:23.115013 22613 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ef453d0a2eb4873b25903242ca39a96 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:23 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:23.115201 1468 tablet_replica.cc:330] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c: stopping tablet replica
I20240502 07:59:23.115290 1469 tablet_replica.cc:330] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:23.115407 1468 raft_consensus.cc:2229] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:23.115715 1469 raft_consensus.cc:2229] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:23.115716 1468 raft_consensus.cc:2258] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:23.115917 1469 raft_consensus.cc:2258] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:23.116504 1469 ts_tablet_manager.cc:1875] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:23.116708 1468 ts_tablet_manager.cc:1875] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20240502 07:59:23.116786 24887 consensus_peers.cc:485] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:23.119165 1468 ts_tablet_manager.cc:1888] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:23.119282 1468 log.cc:1196] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/wals/8ef453d0a2eb4873b25903242ca39a96
I20240502 07:59:23.119335 1469 ts_tablet_manager.cc:1888] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:23.119541 1469 log.cc:1196] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/8ef453d0a2eb4873b25903242ca39a96
I20240502 07:59:23.119642 1468 ts_tablet_manager.cc:1909] T 8ef453d0a2eb4873b25903242ca39a96 P fabddb16c85944648a710ec52fc7244c: Deleting consensus metadata
I20240502 07:59:23.119935 1469 ts_tablet_manager.cc:1909] T 8ef453d0a2eb4873b25903242ca39a96 P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:59:23.120819 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 8ef453d0a2eb4873b25903242ca39a96 (table rebalancer_dynamic_table_new_name [id=1e42a48389cd4b5cb538489ed972801a]) successfully deleted
I20240502 07:59:23.121930 22214 catalog_manager.cc:4830] TS fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): tablet 8ef453d0a2eb4873b25903242ca39a96 (table rebalancer_dynamic_table_new_name [id=1e42a48389cd4b5cb538489ed972801a]) successfully deleted
W20240502 07:59:23.136752 22307 consensus_peers.cc:485] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:23.149173 25019 consensus_peers.cc:485] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:23.178933 24887 consensus_peers.cc:485] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:23.179282 25150 consensus_peers.cc:485] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:23.185487 22569 consensus_peers.cc:485] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:23.216081 22227 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:23.222972 24930 tablet_service.cc:1467] Processing CreateTablet for tablet c24786cd60384a31b52a96041413fbab (DEFAULT_TABLE table=rebalancer_dynamic_table [id=bd1fdb68346a46b8ac2285ee2c456c59]), partition=
I20240502 07:59:23.223075 22351 tablet_service.cc:1467] Processing CreateTablet for tablet c24786cd60384a31b52a96041413fbab (DEFAULT_TABLE table=rebalancer_dynamic_table [id=bd1fdb68346a46b8ac2285ee2c456c59]), partition=
I20240502 07:59:23.223268 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c24786cd60384a31b52a96041413fbab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:23.223311 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c24786cd60384a31b52a96041413fbab. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:23.225219 1407 tablet_bootstrap.cc:492] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:23.225912 1511 tablet_bootstrap.cc:492] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:23.226266 1407 tablet_bootstrap.cc:654] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:23.227226 25063 tablet_service.cc:1467] Processing CreateTablet for tablet c24786cd60384a31b52a96041413fbab (DEFAULT_TABLE table=rebalancer_dynamic_table [id=bd1fdb68346a46b8ac2285ee2c456c59]), partition=
I20240502 07:59:23.227442 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c24786cd60384a31b52a96041413fbab. 1 dirs total, 0 dirs full, 0 dirs failed
W20240502 07:59:23.230515 22307 consensus_peers.cc:485] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:23.232290 1407 tablet_bootstrap.cc:492] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:23.232383 1407 ts_tablet_manager.cc:1367] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:23.232843 1407 raft_consensus.cc:348] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:23.232949 1407 raft_consensus.cc:374] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:23.232985 1407 raft_consensus.cc:729] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:23.233083 1407 consensus_queue.cc:260] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:23.239823 1407 ts_tablet_manager.cc:1398] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
W20240502 07:59:23.244052 25019 consensus_peers.cc:485] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:23.244069 1511 tablet_bootstrap.cc:654] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:23.246546 1511 tablet_bootstrap.cc:492] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:23.246701 1511 ts_tablet_manager.cc:1367] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20240502 07:59:23.247865 1511 raft_consensus.cc:348] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:23.249873 1511 raft_consensus.cc:374] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:23.249969 1511 raft_consensus.cc:729] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:23.250145 1511 consensus_queue.cc:260] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:23.250469 1511 ts_tablet_manager.cc:1398] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
W20240502 07:59:23.252876 22569 consensus_peers.cc:485] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:23.255702 1513 tablet_bootstrap.cc:492] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
W20240502 07:59:23.256351 25150 consensus_peers.cc:485] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:23.256778 1513 tablet_bootstrap.cc:654] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:23.258778 1513 tablet_bootstrap.cc:492] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:23.258872 1513 ts_tablet_manager.cc:1367] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 07:59:23.259310 1513 raft_consensus.cc:348] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:23.259423 1513 raft_consensus.cc:374] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:23.259497 1513 raft_consensus.cc:729] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:23.259641 1513 consensus_queue.cc:260] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:23.259938 1513 ts_tablet_manager.cc:1398] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:23.276165 24930 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:23.282012 25194 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:23.283139 22351 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:23.285398 666 consensus_queue.cc:1035] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:23.291023 22613 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
W20240502 07:59:23.299831 22438 consensus_peers.cc:485] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:23.304139 25063 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:23.313388 22482 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:23.325948 32660 raft_consensus.cc:482] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:23.326057 32660 raft_consensus.cc:504] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:23.326421 32660 leader_election.cc:288] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:23.326874 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c24786cd60384a31b52a96041413fbab" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:23.326972 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c24786cd60384a31b52a96041413fbab" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:23.327011 24951 raft_consensus.cc:2454] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:23.327095 22371 raft_consensus.cc:2454] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 0.
I20240502 07:59:23.327404 25020 leader_election.cc:302] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 227eec4b5f014d349661f6e0b39b9055; no voters:
I20240502 07:59:23.327864 32660 raft_consensus.cc:2781] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:23.327922 32660 raft_consensus.cc:482] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:23.327968 32660 raft_consensus.cc:3037] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:23.328912 32660 raft_consensus.cc:504] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:23.329195 32660 leader_election.cc:288] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:23.329842 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c24786cd60384a31b52a96041413fbab" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:23.329969 24951 raft_consensus.cc:3037] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:23.329948 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c24786cd60384a31b52a96041413fbab" candidate_uuid: "227eec4b5f014d349661f6e0b39b9055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:23.330042 22371 raft_consensus.cc:3037] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:23.331069 22371 raft_consensus.cc:2454] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:23.331069 24951 raft_consensus.cc:2454] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227eec4b5f014d349661f6e0b39b9055 in term 1.
I20240502 07:59:23.332217 25020 leader_election.cc:302] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:23.332918 32660 raft_consensus.cc:2781] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:23.333005 32660 raft_consensus.cc:686] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Leader. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:23.333135 32660 consensus_queue.cc:237] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:23.333844 500 consensus_queue.cc:1035] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:23.337600 22745 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:23.337523 22221 catalog_manager.cc:5484] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135). New cstate: current_term: 1 leader_uuid: "227eec4b5f014d349661f6e0b39b9055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
I20240502 07:59:23.337774 1455 consensus_queue.cc:1035] T 10cdb9745afd4a67b68f8e4e4e14f81b P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 07:59:23.346014 22438 consensus_peers.cc:485] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:23.366025 22700 consensus_peers.cc:462] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:23.404249 492 consensus_queue.cc:1035] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:23.409018 1455 consensus_queue.cc:1035] T 6d605515417c44078b96012144f4110d P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:23.455334 22222 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:23.460569 24951 raft_consensus.cc:1261] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:23.461289 22371 raft_consensus.cc:1261] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Refusing update from remote peer 227eec4b5f014d349661f6e0b39b9055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:23.461860 32660 consensus_queue.cc:1035] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 07:59:23.461942 22569 consensus_peers.cc:462] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:23.461989 365 consensus_queue.cc:1035] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:23.468390 22221 catalog_manager.cc:5576] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995) does not have the latest schema for tablet c24786cd60384a31b52a96041413fbab (table rebalancer_dynamic_table [id=bd1fdb68346a46b8ac2285ee2c456c59]). Expected version 1 got 0
I20240502 07:59:23.469308 22222 catalog_manager.cc:5576] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535) does not have the latest schema for tablet c24786cd60384a31b52a96041413fbab (table rebalancer_dynamic_table [id=bd1fdb68346a46b8ac2285ee2c456c59]). Expected version 1 got 0
W20240502 07:59:23.471010 25150 consensus_peers.cc:462] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:23.473951 22222 catalog_manager.cc:5576] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535) does not have the latest schema for tablet c24786cd60384a31b52a96041413fbab (table rebalancer_dynamic_table [id=bd1fdb68346a46b8ac2285ee2c456c59]). Expected version 1 got 0
I20240502 07:59:23.474814 1582 tablet.cc:1722] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:23.475365 1582 tablet.cc:1620] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:23.476413 1581 tablet.cc:1722] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:23.476800 1581 tablet.cc:1620] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:23.482929 22227 catalog_manager.cc:5576] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995) does not have the latest schema for tablet c24786cd60384a31b52a96041413fbab (table rebalancer_dynamic_table [id=bd1fdb68346a46b8ac2285ee2c456c59]). Expected version 1 got 0
I20240502 07:59:23.484391 22225 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=bd1fdb68346a46b8ac2285ee2c456c59] alter complete (version 1)
I20240502 07:59:23.485098 1583 tablet.cc:1722] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:23.486002 1583 tablet.cc:1620] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:23.527141 25063 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:23.537101 22351 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:23.540771 22613 tablet_service.cc:1429] Tablet server has 25 leaders and 0 scanners
I20240502 07:59:23.543139 22745 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:23.544840 25194 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:23.546898 24930 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:23.569756 22482 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:23.582952 22229 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_66"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:23.596643 22229 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:23.604012 22351 tablet_service.cc:1467] Processing CreateTablet for tablet 22e8725d46084aa5bd30214e77df0960 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=c75d936669f5404aa28a6c446e74f42c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:23.604049 22482 tablet_service.cc:1467] Processing CreateTablet for tablet d21c5d545e894f88a2229110ff4fe8d0 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=c75d936669f5404aa28a6c446e74f42c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:23.604380 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22e8725d46084aa5bd30214e77df0960. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:23.604315 25194 tablet_service.cc:1467] Processing CreateTablet for tablet 22e8725d46084aa5bd30214e77df0960 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=c75d936669f5404aa28a6c446e74f42c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:23.604432 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d21c5d545e894f88a2229110ff4fe8d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:23.604602 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22e8725d46084aa5bd30214e77df0960. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:23.607069 1407 tablet_bootstrap.cc:492] T 22e8725d46084aa5bd30214e77df0960 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:23.608059 24930 tablet_service.cc:1467] Processing CreateTablet for tablet 22e8725d46084aa5bd30214e77df0960 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=c75d936669f5404aa28a6c446e74f42c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:23.608139 22613 tablet_service.cc:1467] Processing CreateTablet for tablet d21c5d545e894f88a2229110ff4fe8d0 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=c75d936669f5404aa28a6c446e74f42c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:23.608368 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22e8725d46084aa5bd30214e77df0960. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:23.608428 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d21c5d545e894f88a2229110ff4fe8d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:23.608918 1407 tablet_bootstrap.cc:654] T 22e8725d46084aa5bd30214e77df0960 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:23.610363 1511 tablet_bootstrap.cc:492] T 22e8725d46084aa5bd30214e77df0960 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:23.610816 22745 tablet_service.cc:1467] Processing CreateTablet for tablet f4f47aca738b4cf3af1a88f6e4d1a57f (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=c75d936669f5404aa28a6c446e74f42c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:23.611066 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4f47aca738b4cf3af1a88f6e4d1a57f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:23.611331 1511 tablet_bootstrap.cc:654] T 22e8725d46084aa5bd30214e77df0960 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
W20240502 07:59:23.612720 25150 consensus_peers.cc:485] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:23.612720 25194 tablet_service.cc:1467] Processing CreateTablet for tablet f4f47aca738b4cf3af1a88f6e4d1a57f (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=c75d936669f5404aa28a6c446e74f42c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:23.613075 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4f47aca738b4cf3af1a88f6e4d1a57f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:23.614135 1511 tablet_bootstrap.cc:492] T 22e8725d46084aa5bd30214e77df0960 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:23.614241 1511 ts_tablet_manager.cc:1367] T 22e8725d46084aa5bd30214e77df0960 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:23.614815 1511 raft_consensus.cc:348] T 22e8725d46084aa5bd30214e77df0960 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:23.614924 1511 raft_consensus.cc:374] T 22e8725d46084aa5bd30214e77df0960 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:23.614957 1511 raft_consensus.cc:729] T 22e8725d46084aa5bd30214e77df0960 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:23.615084 1511 consensus_queue.cc:260] T 22e8725d46084aa5bd30214e77df0960 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:23.615427 1511 ts_tablet_manager.cc:1398] T 22e8725d46084aa5bd30214e77df0960 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:23.616603 1581 tablet.cc:1722] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:23.616920 1581 tablet.cc:1620] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055: MemRowSet was empty: no flush needed.
I20240502 07:59:23.617447 25063 tablet_service.cc:1467] Processing CreateTablet for tablet d21c5d545e894f88a2229110ff4fe8d0 (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=c75d936669f5404aa28a6c446e74f42c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:23.617709 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d21c5d545e894f88a2229110ff4fe8d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:23.619833 1583 tablet.cc:1722] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:23.620100 1583 tablet.cc:1620] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:23.620378 24930 tablet_service.cc:1467] Processing CreateTablet for tablet f4f47aca738b4cf3af1a88f6e4d1a57f (DEFAULT_TABLE table=rebalancer_extra_table_66 [id=c75d936669f5404aa28a6c446e74f42c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:23.620536 1513 tablet_bootstrap.cc:492] T d21c5d545e894f88a2229110ff4fe8d0 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:23.620667 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4f47aca738b4cf3af1a88f6e4d1a57f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:23.621438 1513 tablet_bootstrap.cc:654] T d21c5d545e894f88a2229110ff4fe8d0 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:23.624212 1595 tablet_bootstrap.cc:492] T d21c5d545e894f88a2229110ff4fe8d0 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:23.625404 1595 tablet_bootstrap.cc:654] T d21c5d545e894f88a2229110ff4fe8d0 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:23.625711 1511 tablet_bootstrap.cc:492] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:23.626696 1511 tablet_bootstrap.cc:654] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:23.628372 1511 tablet_bootstrap.cc:492] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:23.628460 1511 ts_tablet_manager.cc:1367] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:23.628980 1511 raft_consensus.cc:348] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:23.629103 1511 raft_consensus.cc:374] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:23.629149 1511 raft_consensus.cc:729] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:23.629274 1511 consensus_queue.cc:260] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:23.629674 1511 ts_tablet_manager.cc:1398] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:23.630162 1582 tablet.cc:1722] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:23.630472 1582 tablet.cc:1620] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:23.634968 22228 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=bd1fdb68346a46b8ac2285ee2c456c59] alter complete (version 2)
I20240502 07:59:23.639937 1595 tablet_bootstrap.cc:492] T d21c5d545e894f88a2229110ff4fe8d0 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:23.640129 1595 ts_tablet_manager.cc:1367] T d21c5d545e894f88a2229110ff4fe8d0 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.002s
I20240502 07:59:23.640821 1595 raft_consensus.cc:348] T d21c5d545e894f88a2229110ff4fe8d0 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:23.640991 1595 raft_consensus.cc:374] T d21c5d545e894f88a2229110ff4fe8d0 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:23.641069 1595 raft_consensus.cc:729] T d21c5d545e894f88a2229110ff4fe8d0 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:23.641237 1595 consensus_queue.cc:260] T d21c5d545e894f88a2229110ff4fe8d0 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:23.642323 1595 ts_tablet_manager.cc:1398] T d21c5d545e894f88a2229110ff4fe8d0 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:23.645573 1598 tablet_bootstrap.cc:492] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:23.646061 1407 tablet_bootstrap.cc:492] T 22e8725d46084aa5bd30214e77df0960 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:23.646163 1407 ts_tablet_manager.cc:1367] T 22e8725d46084aa5bd30214e77df0960 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.000s
I20240502 07:59:23.646898 1407 raft_consensus.cc:348] T 22e8725d46084aa5bd30214e77df0960 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:23.647058 1407 raft_consensus.cc:374] T 22e8725d46084aa5bd30214e77df0960 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:23.647092 1407 raft_consensus.cc:729] T 22e8725d46084aa5bd30214e77df0960 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:23.647212 1407 consensus_queue.cc:260] T 22e8725d46084aa5bd30214e77df0960 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:23.647483 1407 ts_tablet_manager.cc:1398] T 22e8725d46084aa5bd30214e77df0960 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:23.648010 1513 tablet_bootstrap.cc:492] T d21c5d545e894f88a2229110ff4fe8d0 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:23.648110 1513 ts_tablet_manager.cc:1367] T d21c5d545e894f88a2229110ff4fe8d0 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.001s
I20240502 07:59:23.648562 1513 raft_consensus.cc:348] T d21c5d545e894f88a2229110ff4fe8d0 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:23.648671 1513 raft_consensus.cc:374] T d21c5d545e894f88a2229110ff4fe8d0 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:23.648707 1513 raft_consensus.cc:729] T d21c5d545e894f88a2229110ff4fe8d0 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:23.648821 1513 consensus_queue.cc:260] T d21c5d545e894f88a2229110ff4fe8d0 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:23.649119 1513 ts_tablet_manager.cc:1398] T d21c5d545e894f88a2229110ff4fe8d0 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20240502 07:59:23.650638 1600 tablet_bootstrap.cc:492] T f4f47aca738b4cf3af1a88f6e4d1a57f P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:23.651662 1600 tablet_bootstrap.cc:654] T f4f47aca738b4cf3af1a88f6e4d1a57f P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:23.652946 1598 tablet_bootstrap.cc:654] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:23.657586 1596 tablet_bootstrap.cc:492] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:23.658888 1596 tablet_bootstrap.cc:654] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:23.662029 1598 tablet_bootstrap.cc:492] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:23.662446 1598 ts_tablet_manager.cc:1367] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20240502 07:59:23.663004 1598 raft_consensus.cc:348] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:23.663276 1598 raft_consensus.cc:374] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:23.663350 1598 raft_consensus.cc:729] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:23.663524 1598 consensus_queue.cc:260] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:23.663995 1598 ts_tablet_manager.cc:1398] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:23.669903 1024 raft_consensus.cc:482] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:23.670009 1024 raft_consensus.cc:504] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:23.670411 1024 leader_election.cc:288] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:23.670842 1596 tablet_bootstrap.cc:492] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:23.670809 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d21c5d545e894f88a2229110ff4fe8d0" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:23.670924 25084 raft_consensus.cc:2454] T d21c5d545e894f88a2229110ff4fe8d0 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:23.670951 1596 ts_tablet_manager.cc:1367] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 07:59:23.671306 22567 leader_election.cc:302] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:23.671480 1596 raft_consensus.cc:348] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:23.671591 1596 raft_consensus.cc:374] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:23.671628 1596 raft_consensus.cc:729] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:23.671747 1596 consensus_queue.cc:260] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:23.671833 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d21c5d545e894f88a2229110ff4fe8d0" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:23.671942 22501 raft_consensus.cc:2454] T d21c5d545e894f88a2229110ff4fe8d0 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:23.672042 1596 ts_tablet_manager.cc:1398] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:23.672219 1596 tablet_bootstrap.cc:492] T f4f47aca738b4cf3af1a88f6e4d1a57f P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:23.672366 1114 raft_consensus.cc:2781] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:23.672425 1114 raft_consensus.cc:482] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:23.672462 1114 raft_consensus.cc:3037] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:23.673243 1596 tablet_bootstrap.cc:654] T f4f47aca738b4cf3af1a88f6e4d1a57f P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:23.673614 1114 raft_consensus.cc:504] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:23.673933 1114 leader_election.cc:288] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:23.675364 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d21c5d545e894f88a2229110ff4fe8d0" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:23.675524 25084 raft_consensus.cc:3037] T d21c5d545e894f88a2229110ff4fe8d0 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:23.675560 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d21c5d545e894f88a2229110ff4fe8d0" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:23.675670 22501 raft_consensus.cc:3037] T d21c5d545e894f88a2229110ff4fe8d0 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:23.676563 22501 raft_consensus.cc:2454] T d21c5d545e894f88a2229110ff4fe8d0 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:23.676563 25084 raft_consensus.cc:2454] T d21c5d545e894f88a2229110ff4fe8d0 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:23.676847 22567 leader_election.cc:302] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:23.677601 1114 raft_consensus.cc:2781] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:23.677680 1114 raft_consensus.cc:686] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:59:23.677815 1114 consensus_queue.cc:237] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } }
I20240502 07:59:23.678812 25084 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "26b5df5842324628b2e9bb6dc1fc36f2"
dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:52350
I20240502 07:59:23.678957 25084 raft_consensus.cc:595] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Received request to transfer leadership
I20240502 07:59:23.680784 1601 raft_consensus.cc:979] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055: : Instructing follower e51b10af49484aaa8f0267aebc1bf27f to start an election
I20240502 07:59:23.680872 1601 raft_consensus.cc:1067] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Signalling peer e51b10af49484aaa8f0267aebc1bf27f to start an election
I20240502 07:59:23.681525 1600 tablet_bootstrap.cc:492] T f4f47aca738b4cf3af1a88f6e4d1a57f P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:23.681643 22634 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "26b5df5842324628b2e9bb6dc1fc36f2"
dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
from {username='slave'} at 127.15.129.135:39233
I20240502 07:59:23.681696 1600 ts_tablet_manager.cc:1367] T f4f47aca738b4cf3af1a88f6e4d1a57f P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.001s
I20240502 07:59:23.681766 22634 raft_consensus.cc:482] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:23.681843 22634 raft_consensus.cc:3037] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:23.682335 1600 raft_consensus.cc:348] T f4f47aca738b4cf3af1a88f6e4d1a57f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:23.682462 1600 raft_consensus.cc:374] T f4f47aca738b4cf3af1a88f6e4d1a57f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:23.682510 1600 raft_consensus.cc:729] T f4f47aca738b4cf3af1a88f6e4d1a57f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:23.682698 1600 consensus_queue.cc:260] T f4f47aca738b4cf3af1a88f6e4d1a57f P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:23.682902 22634 raft_consensus.cc:504] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:23.683270 1600 ts_tablet_manager.cc:1398] T f4f47aca738b4cf3af1a88f6e4d1a57f P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:23.683291 22634 leader_election.cc:288] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 2 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:23.684639 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b5df5842324628b2e9bb6dc1fc36f2" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:23.684760 25215 raft_consensus.cc:3037] T 26b5df5842324628b2e9bb6dc1fc36f2 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:23.684863 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b5df5842324628b2e9bb6dc1fc36f2" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:23.684955 22764 raft_consensus.cc:3037] T 26b5df5842324628b2e9bb6dc1fc36f2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:23.685750 1596 tablet_bootstrap.cc:492] T f4f47aca738b4cf3af1a88f6e4d1a57f P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:23.685834 1596 ts_tablet_manager.cc:1367] T f4f47aca738b4cf3af1a88f6e4d1a57f P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20240502 07:59:23.685799 25215 raft_consensus.cc:2454] T 26b5df5842324628b2e9bb6dc1fc36f2 P 43930dee0cd5498fa8e357a34dd71432 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 2.
I20240502 07:59:23.685902 22764 raft_consensus.cc:2454] T 26b5df5842324628b2e9bb6dc1fc36f2 P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 2.
I20240502 07:59:23.686187 22570 leader_election.cc:302] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:23.686287 1596 raft_consensus.cc:348] T f4f47aca738b4cf3af1a88f6e4d1a57f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:23.686298 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26b5df5842324628b2e9bb6dc1fc36f2" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:23.686380 1596 raft_consensus.cc:374] T f4f47aca738b4cf3af1a88f6e4d1a57f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:23.686411 1596 raft_consensus.cc:729] T f4f47aca738b4cf3af1a88f6e4d1a57f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:23.686462 25084 raft_consensus.cc:3032] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:23.686511 25084 raft_consensus.cc:729] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Running, Role: LEADER
I20240502 07:59:23.686538 1596 consensus_queue.cc:260] T f4f47aca738b4cf3af1a88f6e4d1a57f P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:23.686672 25084 consensus_queue.cc:260] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:23.686815 1596 ts_tablet_manager.cc:1398] T f4f47aca738b4cf3af1a88f6e4d1a57f P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:23.686859 25084 raft_consensus.cc:3037] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:23.687736 1613 raft_consensus.cc:2781] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:23.687786 25084 raft_consensus.cc:2454] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 2.
I20240502 07:59:23.688880 1613 raft_consensus.cc:686] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [term 2 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:59:23.689271 1613 consensus_queue.cc:237] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:23.689241 22228 catalog_manager.cc:5484] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } }
I20240502 07:59:23.694178 22229 catalog_manager.cc:5484] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 1 to 2, leader changed from 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 2 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:23.740351 22229 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:23.746922 1581 tablet.cc:1722] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:23.747458 1582 tablet.cc:1722] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:23.752077 1583 tablet.cc:1722] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:23.757108 22229 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=bd1fdb68346a46b8ac2285ee2c456c59] alter complete (version 3)
W20240502 07:59:23.816690 22700 consensus_peers.cc:485] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:23.842746 601 raft_consensus.cc:482] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:23.842880 601 raft_consensus.cc:504] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
W20240502 07:59:23.843019 22700 consensus_peers.cc:462] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:23.843267 601 leader_election.cc:288] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:23.843256 492 raft_consensus.cc:482] T f4f47aca738b4cf3af1a88f6e4d1a57f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:23.843376 492 raft_consensus.cc:504] T f4f47aca738b4cf3af1a88f6e4d1a57f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:23.843662 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4f47aca738b4cf3af1a88f6e4d1a57f" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:23.843734 492 leader_election.cc:288] T f4f47aca738b4cf3af1a88f6e4d1a57f P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:23.843784 25215 raft_consensus.cc:2454] T f4f47aca738b4cf3af1a88f6e4d1a57f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:23.844008 24888 leader_election.cc:302] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:23.844173 601 raft_consensus.cc:2781] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:23.844231 601 raft_consensus.cc:482] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:23.844278 601 raft_consensus.cc:3037] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:23.844492 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4f47aca738b4cf3af1a88f6e4d1a57f" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:23.844617 22764 raft_consensus.cc:2454] T f4f47aca738b4cf3af1a88f6e4d1a57f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:23.845280 601 raft_consensus.cc:504] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:23.845495 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4f47aca738b4cf3af1a88f6e4d1a57f" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:23.845624 24951 raft_consensus.cc:2379] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 43930dee0cd5498fa8e357a34dd71432 in current term 1: Already voted for candidate 1aecef81733c47b3b468504934357231 in this term.
I20240502 07:59:23.845643 601 leader_election.cc:288] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591), f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673)
I20240502 07:59:23.846012 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4f47aca738b4cf3af1a88f6e4d1a57f" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:23.846071 22764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4f47aca738b4cf3af1a88f6e4d1a57f" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:23.846113 25215 raft_consensus.cc:3037] T f4f47aca738b4cf3af1a88f6e4d1a57f P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:23.846184 22764 raft_consensus.cc:3037] T f4f47aca738b4cf3af1a88f6e4d1a57f P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:23.847115 22764 raft_consensus.cc:2454] T f4f47aca738b4cf3af1a88f6e4d1a57f P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:23.847116 25215 raft_consensus.cc:2454] T f4f47aca738b4cf3af1a88f6e4d1a57f P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:23.847337 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4f47aca738b4cf3af1a88f6e4d1a57f" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:23.847460 22763 raft_consensus.cc:2379] T f4f47aca738b4cf3af1a88f6e4d1a57f P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 43930dee0cd5498fa8e357a34dd71432 in current term 1: Already voted for candidate 1aecef81733c47b3b468504934357231 in this term.
I20240502 07:59:23.847568 24888 leader_election.cc:302] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:23.847707 25151 leader_election.cc:302] T f4f47aca738b4cf3af1a88f6e4d1a57f P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432; no voters: 1aecef81733c47b3b468504934357231, f00f3a1876444ad2bf9c919068ab8014
I20240502 07:59:23.847858 492 raft_consensus.cc:2726] T f4f47aca738b4cf3af1a88f6e4d1a57f P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20240502 07:59:23.847939 601 raft_consensus.cc:2781] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:23.848022 601 raft_consensus.cc:686] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:23.848170 601 consensus_queue.cc:237] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:23.850687 22222 catalog_manager.cc:5484] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:23.859844 22222 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:23.859970 22222 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:23.861972 22222 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet c24786cd60384a31b52a96041413fbab
I20240502 07:59:23.862675 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet c24786cd60384a31b52a96041413fbab with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:23 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:23.862715 24930 tablet_service.cc:1514] Processing DeleteTablet for tablet c24786cd60384a31b52a96041413fbab with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:23 UTC) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:23.863065 1656 tablet_replica.cc:330] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:23.863121 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet c24786cd60384a31b52a96041413fbab with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:23 UTC) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:23.863255 1656 raft_consensus.cc:2229] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:23.863418 1656 raft_consensus.cc:2258] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:23.863531 1655 tablet_replica.cc:330] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:23.863591 1657 tablet_replica.cc:330] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:23.863772 1657 raft_consensus.cc:2229] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:23.864018 1657 raft_consensus.cc:2258] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:23.864223 1655 raft_consensus.cc:2229] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:23.864404 1655 raft_consensus.cc:2258] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:23.864425 1656 ts_tablet_manager.cc:1875] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:23.864925 1657 ts_tablet_manager.cc:1875] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:23.865278 1655 ts_tablet_manager.cc:1875] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:23.867743 1656 ts_tablet_manager.cc:1888] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:23.867900 1656 log.cc:1196] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/c24786cd60384a31b52a96041413fbab
I20240502 07:59:23.867879 1655 ts_tablet_manager.cc:1888] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:23.868059 1655 log.cc:1196] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/c24786cd60384a31b52a96041413fbab
I20240502 07:59:23.868350 1657 ts_tablet_manager.cc:1888] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:23.868415 1655 ts_tablet_manager.cc:1909] T c24786cd60384a31b52a96041413fbab P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:59:23.868422 1657 log.cc:1196] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/c24786cd60384a31b52a96041413fbab
I20240502 07:59:23.868427 1656 ts_tablet_manager.cc:1909] T c24786cd60384a31b52a96041413fbab P 1aecef81733c47b3b468504934357231: Deleting consensus metadata
I20240502 07:59:23.868801 1657 ts_tablet_manager.cc:1909] T c24786cd60384a31b52a96041413fbab P 227eec4b5f014d349661f6e0b39b9055: Deleting consensus metadata
I20240502 07:59:23.869131 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet c24786cd60384a31b52a96041413fbab (table rebalancer_dynamic_table_new_name [id=bd1fdb68346a46b8ac2285ee2c456c59]) successfully deleted
I20240502 07:59:23.869371 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet c24786cd60384a31b52a96041413fbab (table rebalancer_dynamic_table_new_name [id=bd1fdb68346a46b8ac2285ee2c456c59]) successfully deleted
I20240502 07:59:23.869511 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet c24786cd60384a31b52a96041413fbab (table rebalancer_dynamic_table_new_name [id=bd1fdb68346a46b8ac2285ee2c456c59]) successfully deleted
I20240502 07:59:23.877032 492 raft_consensus.cc:482] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:23.877157 492 raft_consensus.cc:504] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:23.877609 492 leader_election.cc:288] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:23.878244 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22e8725d46084aa5bd30214e77df0960" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:23.878266 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22e8725d46084aa5bd30214e77df0960" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:23.878388 24951 raft_consensus.cc:2454] T 22e8725d46084aa5bd30214e77df0960 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:23.878404 22371 raft_consensus.cc:2454] T 22e8725d46084aa5bd30214e77df0960 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 0.
I20240502 07:59:23.878729 25151 leader_election.cc:302] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43930dee0cd5498fa8e357a34dd71432, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:23.879014 492 raft_consensus.cc:2781] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:23.879068 492 raft_consensus.cc:482] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:23.879124 492 raft_consensus.cc:3037] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:23.879979 492 raft_consensus.cc:504] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:23.880290 492 leader_election.cc:288] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Requested vote from peers b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:23.880602 22371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22e8725d46084aa5bd30214e77df0960" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
I20240502 07:59:23.880652 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22e8725d46084aa5bd30214e77df0960" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:23.880710 22371 raft_consensus.cc:3037] T 22e8725d46084aa5bd30214e77df0960 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:23.880792 24951 raft_consensus.cc:3037] T 22e8725d46084aa5bd30214e77df0960 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:23.881642 24951 raft_consensus.cc:2454] T 22e8725d46084aa5bd30214e77df0960 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:23.881736 22371 raft_consensus.cc:2454] T 22e8725d46084aa5bd30214e77df0960 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
I20240502 07:59:23.881898 25151 leader_election.cc:302] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:23.882081 492 raft_consensus.cc:2781] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:23.882181 492 raft_consensus.cc:686] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [term 1 LEADER]: Becoming Leader. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Running, Role: LEADER
I20240502 07:59:23.882349 492 consensus_queue.cc:237] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:23.885066 22222 catalog_manager.cc:5484] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136). New cstate: current_term: 1 leader_uuid: "43930dee0cd5498fa8e357a34dd71432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:23.910301 25063 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:23.915236 24930 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:23.918557 25194 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:23.921485 22351 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:23.940081 22613 tablet_service.cc:1429] Tablet server has 27 leaders and 0 scanners
I20240502 07:59:23.946314 22482 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:23.964360 22222 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:23.965858 22745 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:23.973767 22745 tablet_service.cc:1467] Processing CreateTablet for tablet dac19fc2557248cc900081045ecea2a9 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=142b302e127f40e4808828e001274c13]), partition=
I20240502 07:59:23.973989 22351 tablet_service.cc:1467] Processing CreateTablet for tablet dac19fc2557248cc900081045ecea2a9 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=142b302e127f40e4808828e001274c13]), partition=
I20240502 07:59:23.974092 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dac19fc2557248cc900081045ecea2a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:23.974134 24930 tablet_service.cc:1467] Processing CreateTablet for tablet dac19fc2557248cc900081045ecea2a9 (DEFAULT_TABLE table=rebalancer_dynamic_table [id=142b302e127f40e4808828e001274c13]), partition=
I20240502 07:59:23.974247 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dac19fc2557248cc900081045ecea2a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:23.974381 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dac19fc2557248cc900081045ecea2a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:23.976361 1511 tablet_bootstrap.cc:492] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:23.976431 1600 tablet_bootstrap.cc:492] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:23.976593 1407 tablet_bootstrap.cc:492] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:23.977375 1511 tablet_bootstrap.cc:654] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:23.977468 1600 tablet_bootstrap.cc:654] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:23.977516 1407 tablet_bootstrap.cc:654] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:23.978811 1407 tablet_bootstrap.cc:492] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:23.978888 1407 ts_tablet_manager.cc:1367] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:23.979316 1407 raft_consensus.cc:348] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:23.979435 1407 raft_consensus.cc:374] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:23.979476 1407 raft_consensus.cc:729] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:23.979593 1407 consensus_queue.cc:260] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:23.979895 1407 ts_tablet_manager.cc:1398] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:23.981762 1600 tablet_bootstrap.cc:492] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:23.981874 1600 ts_tablet_manager.cc:1367] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20240502 07:59:23.982367 1600 raft_consensus.cc:348] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:23.982470 1600 raft_consensus.cc:374] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:23.982501 1600 raft_consensus.cc:729] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:23.982623 1600 consensus_queue.cc:260] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:23.982925 1600 ts_tablet_manager.cc:1398] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20240502 07:59:23.994666 25150 consensus_peers.cc:462] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:24.002792 1511 tablet_bootstrap.cc:492] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:24.002911 1511 ts_tablet_manager.cc:1367] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20240502 07:59:24.003566 1511 raft_consensus.cc:348] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:24.003687 1511 raft_consensus.cc:374] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.003719 1511 raft_consensus.cc:729] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.003854 1511 consensus_queue.cc:260] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:24.004122 1511 ts_tablet_manager.cc:1398] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
W20240502 07:59:24.032493 22569 consensus_peers.cc:462] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:24.079576 1613 consensus_queue.cc:1035] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:24.088410 1615 consensus_queue.cc:1035] T d21c5d545e894f88a2229110ff4fe8d0 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:24.094254 32390 raft_consensus.cc:482] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:24.094367 32390 raft_consensus.cc:504] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:24.094661 25215 raft_consensus.cc:1261] T 26b5df5842324628b2e9bb6dc1fc36f2 P 43930dee0cd5498fa8e357a34dd71432 [term 2 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 07:59:24.094763 32390 leader_election.cc:288] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:24.096395 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dac19fc2557248cc900081045ecea2a9" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:24.096539 24951 raft_consensus.cc:2454] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:24.096917 22308 leader_election.cc:302] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:24.097100 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dac19fc2557248cc900081045ecea2a9" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:24.097236 22763 raft_consensus.cc:2454] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:24.097501 1615 consensus_queue.cc:1035] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:24.099192 32390 raft_consensus.cc:2781] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:24.099252 32390 raft_consensus.cc:482] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:24.099290 32390 raft_consensus.cc:3037] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.100385 32390 raft_consensus.cc:504] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:24.100693 32390 leader_election.cc:288] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:24.100948 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dac19fc2557248cc900081045ecea2a9" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:24.101006 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dac19fc2557248cc900081045ecea2a9" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:24.101054 22763 raft_consensus.cc:3037] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.101116 24951 raft_consensus.cc:3037] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.102039 22763 raft_consensus.cc:2454] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:24.102039 24951 raft_consensus.cc:2454] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:24.102501 22308 leader_election.cc:302] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:24.102778 308 raft_consensus.cc:2781] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:24.102854 308 raft_consensus.cc:686] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:59:24.102977 308 consensus_queue.cc:237] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
W20240502 07:59:24.104414 22307 consensus_peers.cc:485] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:24.105500 22224 catalog_manager.cc:5484] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
W20240502 07:59:24.115664 22216 catalog_manager.cc:4573] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): CreateTablet RPC failed for tablet 7c09dfefb9c246c684884973bdc35c75: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:24.130314 22700 consensus_peers.cc:485] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:24.151203 24930 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:24.164000 22613 tablet_service.cc:1429] Tablet server has 27 leaders and 0 scanners
I20240502 07:59:24.165217 25194 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:24.165498 22351 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 07:59:24.166204 22482 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:24.170212 25063 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:24.171936 22745 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:24.226564 22228 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:24.227603 22763 raft_consensus.cc:1261] T 26b5df5842324628b2e9bb6dc1fc36f2 P f00f3a1876444ad2bf9c919068ab8014 [term 2 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 07:59:24.228145 1024 consensus_queue.cc:1035] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:24.235595 24951 raft_consensus.cc:1261] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:24.236354 308 consensus_queue.cc:1035] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:24.238896 22763 raft_consensus.cc:1261] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:24.241904 25084 raft_consensus.cc:1261] T 26b5df5842324628b2e9bb6dc1fc36f2 P 227eec4b5f014d349661f6e0b39b9055 [term 2 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20240502 07:59:24.242204 308 consensus_queue.cc:1035] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:24.242595 22225 catalog_manager.cc:5576] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995) does not have the latest schema for tablet dac19fc2557248cc900081045ecea2a9 (table rebalancer_dynamic_table [id=142b302e127f40e4808828e001274c13]). Expected version 1 got 0
I20240502 07:59:24.246073 1615 consensus_queue.cc:1035] T 26b5df5842324628b2e9bb6dc1fc36f2 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:24.247706 22225 catalog_manager.cc:5576] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673) does not have the latest schema for tablet dac19fc2557248cc900081045ecea2a9 (table rebalancer_dynamic_table [id=142b302e127f40e4808828e001274c13]). Expected version 1 got 0
I20240502 07:59:24.247843 1582 tablet.cc:1722] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:24.248229 1582 tablet.cc:1620] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:24.248216 1583 tablet.cc:1722] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:24.248500 1583 tablet.cc:1620] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:24.253449 1751 tablet.cc:1722] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:24.253895 1751 tablet.cc:1620] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:59:24.254866 601 consensus_queue.cc:1035] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:24.259245 22228 catalog_manager.cc:5576] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673) does not have the latest schema for tablet dac19fc2557248cc900081045ecea2a9 (table rebalancer_dynamic_table [id=142b302e127f40e4808828e001274c13]). Expected version 1 got 0
I20240502 07:59:24.259809 22225 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=142b302e127f40e4808828e001274c13] alter complete (version 1)
I20240502 07:59:24.263186 32485 consensus_queue.cc:1035] T f4f47aca738b4cf3af1a88f6e4d1a57f P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 07:59:24.278072 25150 consensus_peers.cc:485] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:24.365797 22223 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:24.382205 1582 tablet.cc:1722] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:24.382577 1582 tablet.cc:1620] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:24.384138 25063 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:24.387945 1751 tablet.cc:1722] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:24.388244 1751 tablet.cc:1620] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014: MemRowSet was empty: no flush needed.
I20240502 07:59:24.388404 1583 tablet.cc:1722] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:24.388684 1583 tablet.cc:1620] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:24.390419 25194 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:24.395241 22224 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20240502 07:59:24.396261 22700 consensus_peers.cc:462] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:24.396724 492 consensus_queue.cc:1035] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:24.400028 1455 consensus_queue.cc:1035] T 22e8725d46084aa5bd30214e77df0960 P 43930dee0cd5498fa8e357a34dd71432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 07:59:24.405689 22438 consensus_peers.cc:485] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:24.406287 22222 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=142b302e127f40e4808828e001274c13] alter complete (version 2)
I20240502 07:59:24.410310 22613 tablet_service.cc:1429] Tablet server has 27 leaders and 0 scanners
I20240502 07:59:24.410914 22351 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 07:59:24.412341 24930 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:24.414755 22482 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:24.425750 22745 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:24.431565 25063 tablet_service.cc:1467] Processing CreateTablet for tablet c3a4b31595124c2681d6430c4748d3ff (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=f5f40aff97934c1d804a821f1a5fce70]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.431890 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3a4b31595124c2681d6430c4748d3ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.431865 25194 tablet_service.cc:1467] Processing CreateTablet for tablet c3a4b31595124c2681d6430c4748d3ff (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=f5f40aff97934c1d804a821f1a5fce70]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.432049 22482 tablet_service.cc:1467] Processing CreateTablet for tablet cdc3697b2b8f4306ae0901b650e334b3 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=f5f40aff97934c1d804a821f1a5fce70]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.432076 25193 tablet_service.cc:1467] Processing CreateTablet for tablet cdc3697b2b8f4306ae0901b650e334b3 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=f5f40aff97934c1d804a821f1a5fce70]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.432133 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3a4b31595124c2681d6430c4748d3ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.432294 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdc3697b2b8f4306ae0901b650e334b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.434430 25193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdc3697b2b8f4306ae0901b650e334b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.436570 22351 tablet_service.cc:1467] Processing CreateTablet for tablet c3a4b31595124c2681d6430c4748d3ff (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=f5f40aff97934c1d804a821f1a5fce70]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.436846 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3a4b31595124c2681d6430c4748d3ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.438896 1407 tablet_bootstrap.cc:492] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:24.439841 1407 tablet_bootstrap.cc:654] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.440433 1797 tablet_bootstrap.cc:492] T c3a4b31595124c2681d6430c4748d3ff P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:24.441475 22613 tablet_service.cc:1467] Processing CreateTablet for tablet b2e55f41eb174341bea30ccee0f50e15 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=f5f40aff97934c1d804a821f1a5fce70]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.441475 24930 tablet_service.cc:1467] Processing CreateTablet for tablet cdc3697b2b8f4306ae0901b650e334b3 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=f5f40aff97934c1d804a821f1a5fce70]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.441725 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdc3697b2b8f4306ae0901b650e334b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.441747 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2e55f41eb174341bea30ccee0f50e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.443821 1511 tablet_bootstrap.cc:492] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:24.444048 1799 tablet_bootstrap.cc:492] T cdc3697b2b8f4306ae0901b650e334b3 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:24.444272 22745 tablet_service.cc:1467] Processing CreateTablet for tablet b2e55f41eb174341bea30ccee0f50e15 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=f5f40aff97934c1d804a821f1a5fce70]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.444537 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2e55f41eb174341bea30ccee0f50e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.444698 1511 tablet_bootstrap.cc:654] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.444926 1799 tablet_bootstrap.cc:654] T cdc3697b2b8f4306ae0901b650e334b3 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.446589 1600 tablet_bootstrap.cc:492] T b2e55f41eb174341bea30ccee0f50e15 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:24.447320 22482 tablet_service.cc:1467] Processing CreateTablet for tablet b2e55f41eb174341bea30ccee0f50e15 (DEFAULT_TABLE table=rebalancer_extra_table_68 [id=f5f40aff97934c1d804a821f1a5fce70]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:24.447440 1600 tablet_bootstrap.cc:654] T b2e55f41eb174341bea30ccee0f50e15 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.447569 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2e55f41eb174341bea30ccee0f50e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.448180 1797 tablet_bootstrap.cc:654] T c3a4b31595124c2681d6430c4748d3ff P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
W20240502 07:59:24.448858 25019 consensus_peers.cc:485] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:24.448920 25019 consensus_peers.cc:485] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:24.449680 1798 tablet_bootstrap.cc:492] T c3a4b31595124c2681d6430c4748d3ff P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:24.452193 1798 tablet_bootstrap.cc:654] T c3a4b31595124c2681d6430c4748d3ff P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.454007 1600 tablet_bootstrap.cc:492] T b2e55f41eb174341bea30ccee0f50e15 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:24.454093 1600 ts_tablet_manager.cc:1367] T b2e55f41eb174341bea30ccee0f50e15 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20240502 07:59:24.454530 1600 raft_consensus.cc:348] T b2e55f41eb174341bea30ccee0f50e15 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:24.454648 1600 raft_consensus.cc:374] T b2e55f41eb174341bea30ccee0f50e15 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.454671 1801 tablet_bootstrap.cc:492] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:24.454684 1600 raft_consensus.cc:729] T b2e55f41eb174341bea30ccee0f50e15 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.454814 1600 consensus_queue.cc:260] T b2e55f41eb174341bea30ccee0f50e15 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:24.454913 1407 tablet_bootstrap.cc:492] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:24.454995 1407 ts_tablet_manager.cc:1367] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 07:59:24.455133 1600 ts_tablet_manager.cc:1398] T b2e55f41eb174341bea30ccee0f50e15 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:24.455437 1407 raft_consensus.cc:348] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:24.455544 1407 raft_consensus.cc:374] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.455577 1407 raft_consensus.cc:729] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.455685 1407 consensus_queue.cc:260] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:24.455824 1801 tablet_bootstrap.cc:654] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.455932 1407 ts_tablet_manager.cc:1398] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:24.457688 1797 tablet_bootstrap.cc:492] T c3a4b31595124c2681d6430c4748d3ff P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:24.458171 1797 ts_tablet_manager.cc:1367] T c3a4b31595124c2681d6430c4748d3ff P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 07:59:24.458626 1797 raft_consensus.cc:348] T c3a4b31595124c2681d6430c4748d3ff P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:24.458784 1797 raft_consensus.cc:374] T c3a4b31595124c2681d6430c4748d3ff P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.458850 1797 raft_consensus.cc:729] T c3a4b31595124c2681d6430c4748d3ff P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.459008 1797 consensus_queue.cc:260] T c3a4b31595124c2681d6430c4748d3ff P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:24.459563 1797 ts_tablet_manager.cc:1398] T c3a4b31595124c2681d6430c4748d3ff P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:24.460412 1798 tablet_bootstrap.cc:492] T c3a4b31595124c2681d6430c4748d3ff P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:24.460546 1798 ts_tablet_manager.cc:1367] T c3a4b31595124c2681d6430c4748d3ff P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20240502 07:59:24.461009 1798 raft_consensus.cc:348] T c3a4b31595124c2681d6430c4748d3ff P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:24.461171 1798 raft_consensus.cc:374] T c3a4b31595124c2681d6430c4748d3ff P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.461236 1798 raft_consensus.cc:729] T c3a4b31595124c2681d6430c4748d3ff P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.461428 1798 consensus_queue.cc:260] T c3a4b31595124c2681d6430c4748d3ff P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:24.461735 1798 ts_tablet_manager.cc:1398] T c3a4b31595124c2681d6430c4748d3ff P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:24.462680 1798 tablet_bootstrap.cc:492] T cdc3697b2b8f4306ae0901b650e334b3 P 43930dee0cd5498fa8e357a34dd71432: Bootstrap starting.
I20240502 07:59:24.463621 1798 tablet_bootstrap.cc:654] T cdc3697b2b8f4306ae0901b650e334b3 P 43930dee0cd5498fa8e357a34dd71432: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.464620 1798 tablet_bootstrap.cc:492] T cdc3697b2b8f4306ae0901b650e334b3 P 43930dee0cd5498fa8e357a34dd71432: No bootstrap required, opened a new log
I20240502 07:59:24.464699 1798 ts_tablet_manager.cc:1367] T cdc3697b2b8f4306ae0901b650e334b3 P 43930dee0cd5498fa8e357a34dd71432: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:24.465078 1798 raft_consensus.cc:348] T cdc3697b2b8f4306ae0901b650e334b3 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:24.465188 1798 raft_consensus.cc:374] T cdc3697b2b8f4306ae0901b650e334b3 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.465215 1798 raft_consensus.cc:729] T cdc3697b2b8f4306ae0901b650e334b3 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43930dee0cd5498fa8e357a34dd71432, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.465309 1798 consensus_queue.cc:260] T cdc3697b2b8f4306ae0901b650e334b3 P 43930dee0cd5498fa8e357a34dd71432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:24.465566 1798 ts_tablet_manager.cc:1398] T cdc3697b2b8f4306ae0901b650e334b3 P 43930dee0cd5498fa8e357a34dd71432: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20240502 07:59:24.473182 25150 consensus_peers.cc:485] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:24.483465 25019 consensus_peers.cc:485] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:24.490476 1799 tablet_bootstrap.cc:492] T cdc3697b2b8f4306ae0901b650e334b3 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:24.490484 1511 tablet_bootstrap.cc:492] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:24.490577 1799 ts_tablet_manager.cc:1367] T cdc3697b2b8f4306ae0901b650e334b3 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.000s
I20240502 07:59:24.490581 1511 ts_tablet_manager.cc:1367] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.000s
I20240502 07:59:24.491019 1799 raft_consensus.cc:348] T cdc3697b2b8f4306ae0901b650e334b3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:24.491041 1511 raft_consensus.cc:348] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:24.491134 1799 raft_consensus.cc:374] T cdc3697b2b8f4306ae0901b650e334b3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.491142 1511 raft_consensus.cc:374] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.491163 1799 raft_consensus.cc:729] T cdc3697b2b8f4306ae0901b650e334b3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.491170 1511 raft_consensus.cc:729] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.491269 1511 consensus_queue.cc:260] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:24.491269 1799 consensus_queue.cc:260] T cdc3697b2b8f4306ae0901b650e334b3 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:24.491524 1511 ts_tablet_manager.cc:1398] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:24.491535 1799 ts_tablet_manager.cc:1398] T cdc3697b2b8f4306ae0901b650e334b3 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:24.491681 1799 tablet_bootstrap.cc:492] T b2e55f41eb174341bea30ccee0f50e15 P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:24.491988 1801 tablet_bootstrap.cc:492] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:24.492067 1801 ts_tablet_manager.cc:1367] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.000s
I20240502 07:59:24.492473 1801 raft_consensus.cc:348] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:24.492566 1801 raft_consensus.cc:374] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.492594 1801 raft_consensus.cc:729] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.492638 1799 tablet_bootstrap.cc:654] T b2e55f41eb174341bea30ccee0f50e15 P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.492693 1801 consensus_queue.cc:260] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:24.492951 1801 ts_tablet_manager.cc:1398] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:24.495857 308 raft_consensus.cc:482] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:24.495951 308 raft_consensus.cc:504] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:24.496341 308 leader_election.cc:288] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:24.496655 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3a4b31595124c2681d6430c4748d3ff" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:24.496766 25084 raft_consensus.cc:2454] T c3a4b31595124c2681d6430c4748d3ff P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:24.497027 22305 leader_election.cc:302] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:24.497148 1799 tablet_bootstrap.cc:492] T b2e55f41eb174341bea30ccee0f50e15 P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:24.497215 1799 ts_tablet_manager.cc:1367] T b2e55f41eb174341bea30ccee0f50e15 P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 07:59:24.497627 1799 raft_consensus.cc:348] T b2e55f41eb174341bea30ccee0f50e15 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:24.497759 1799 raft_consensus.cc:374] T b2e55f41eb174341bea30ccee0f50e15 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.497799 1799 raft_consensus.cc:729] T b2e55f41eb174341bea30ccee0f50e15 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.497833 32390 raft_consensus.cc:2781] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:24.497889 32390 raft_consensus.cc:482] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:24.497922 32390 raft_consensus.cc:3037] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.497947 1799 consensus_queue.cc:260] T b2e55f41eb174341bea30ccee0f50e15 P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:24.498270 1799 ts_tablet_manager.cc:1398] T b2e55f41eb174341bea30ccee0f50e15 P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:24.498857 32390 raft_consensus.cc:504] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:24.499157 32390 leader_election.cc:288] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:24.499471 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3a4b31595124c2681d6430c4748d3ff" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:24.499563 25084 raft_consensus.cc:3037] T c3a4b31595124c2681d6430c4748d3ff P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.500424 25084 raft_consensus.cc:2454] T c3a4b31595124c2681d6430c4748d3ff P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:24.500682 22305 leader_election.cc:302] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:24.505070 308 raft_consensus.cc:2781] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:24.505182 308 raft_consensus.cc:686] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:59:24.505300 308 consensus_queue.cc:237] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:24.507828 22222 catalog_manager.cc:5484] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:24.511293 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3a4b31595124c2681d6430c4748d3ff" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:24.511425 25215 raft_consensus.cc:3037] T c3a4b31595124c2681d6430c4748d3ff P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.512236 25213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3a4b31595124c2681d6430c4748d3ff" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:24.512396 25215 raft_consensus.cc:2454] T c3a4b31595124c2681d6430c4748d3ff P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:24.514509 32485 raft_consensus.cc:482] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:24.514595 32485 raft_consensus.cc:504] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:24.514925 32485 leader_election.cc:288] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:24.515781 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdc3697b2b8f4306ae0901b650e334b3" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432" is_pre_election: true
I20240502 07:59:24.515811 22499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdc3697b2b8f4306ae0901b650e334b3" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:24.515901 25215 raft_consensus.cc:2454] T cdc3697b2b8f4306ae0901b650e334b3 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:24.515923 22499 raft_consensus.cc:2454] T cdc3697b2b8f4306ae0901b650e334b3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:24.516196 24885 leader_election.cc:302] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:24.516564 1673 raft_consensus.cc:2781] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:24.516618 1673 raft_consensus.cc:482] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:24.516660 1673 raft_consensus.cc:3037] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.517655 1673 raft_consensus.cc:504] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:24.517958 1673 leader_election.cc:288] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625), 43930dee0cd5498fa8e357a34dd71432 (127.15.129.136:36591)
I20240502 07:59:24.518451 25215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdc3697b2b8f4306ae0901b650e334b3" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43930dee0cd5498fa8e357a34dd71432"
I20240502 07:59:24.518550 25215 raft_consensus.cc:3037] T cdc3697b2b8f4306ae0901b650e334b3 P 43930dee0cd5498fa8e357a34dd71432 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.519425 25215 raft_consensus.cc:2454] T cdc3697b2b8f4306ae0901b650e334b3 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:24.519685 22222 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
I20240502 07:59:24.528200 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdc3697b2b8f4306ae0901b650e334b3" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:24.528345 22501 raft_consensus.cc:3037] T cdc3697b2b8f4306ae0901b650e334b3 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.529251 22501 raft_consensus.cc:2454] T cdc3697b2b8f4306ae0901b650e334b3 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:24.529844 24888 leader_election.cc:302] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, 43930dee0cd5498fa8e357a34dd71432; no voters:
I20240502 07:59:24.530345 32485 raft_consensus.cc:2781] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:24.530440 32485 raft_consensus.cc:686] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:24.530575 32485 consensus_queue.cc:237] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:24.532986 1582 tablet.cc:1722] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:24.533360 1583 tablet.cc:1722] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:24.533566 22222 catalog_manager.cc:5484] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:24.536013 1751 tablet.cc:1722] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
W20240502 07:59:24.536870 25019 consensus_peers.cc:485] T 5034380b5c0846faa1494140d67203cc P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:24.539729 25150 consensus_peers.cc:462] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:24.540596 22223 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=142b302e127f40e4808828e001274c13] alter complete (version 3)
I20240502 07:59:24.561245 22371 consensus_queue.cc:237] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:24.562628 25084 raft_consensus.cc:1261] T ff6bc97390164a37bcdaafd651c888cf P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:24.562688 22501 raft_consensus.cc:1261] T ff6bc97390164a37bcdaafd651c888cf P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:24.562975 1740 consensus_queue.cc:1035] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:24.563203 32390 consensus_queue.cc:1035] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:24.564781 1740 raft_consensus.cc:2932] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:24.565172 25084 raft_consensus.cc:2932] T ff6bc97390164a37bcdaafd651c888cf P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:24.565848 22501 raft_consensus.cc:2932] T ff6bc97390164a37bcdaafd651c888cf P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:24.571048 22222 catalog_manager.cc:5484] T ff6bc97390164a37bcdaafd651c888cf P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:24.573621 24930 tablet_service.cc:1514] Processing DeleteTablet for tablet ff6bc97390164a37bcdaafd651c888cf with delete_type TABLET_DATA_TOMBSTONED (TS 1aecef81733c47b3b468504934357231 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:24.575614 1835 tablet_replica.cc:330] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:24.577046 1835 raft_consensus.cc:2229] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:24.577231 1835 raft_consensus.cc:2258] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:24.577772 1835 ts_tablet_manager.cc:1875] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:24.582768 1835 ts_tablet_manager.cc:1888] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:24.582851 1835 log.cc:1196] T ff6bc97390164a37bcdaafd651c888cf P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/ff6bc97390164a37bcdaafd651c888cf
I20240502 07:59:24.583359 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet ff6bc97390164a37bcdaafd651c888cf (table rebalancer_extra_table_36 [id=3c759007a4b047acbe8f33665e1018fc]) successfully deleted
W20240502 07:59:24.598441 22569 consensus_peers.cc:462] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:24.614056 25019 consensus_peers.cc:485] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:24.632827 22307 consensus_peers.cc:485] T 10ae14d069e048ccba7fdfc79951cec0 P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:24.635010 22700 consensus_peers.cc:485] T 01d9a73be1fd472db39860e538b9c97e P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:24.646497 22223 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:24.646646 22223 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:24.648888 22223 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet dac19fc2557248cc900081045ecea2a9
I20240502 07:59:24.649390 22745 tablet_service.cc:1514] Processing DeleteTablet for tablet dac19fc2557248cc900081045ecea2a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:24 UTC) from {username='slave'} at 127.0.0.1:49266
I20240502 07:59:24.649724 24930 tablet_service.cc:1514] Processing DeleteTablet for tablet dac19fc2557248cc900081045ecea2a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:24 UTC) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:24.649916 1835 tablet_replica.cc:330] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:24.650043 1835 raft_consensus.cc:2229] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:24.650151 1835 raft_consensus.cc:2258] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:24.650334 22351 tablet_service.cc:1514] Processing DeleteTablet for tablet dac19fc2557248cc900081045ecea2a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:24 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:24.650658 1873 tablet_replica.cc:330] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:24.650797 1873 raft_consensus.cc:2229] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:24.651194 1835 ts_tablet_manager.cc:1875] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:24.651388 1873 raft_consensus.cc:2258] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:24.651970 1873 ts_tablet_manager.cc:1875] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
W20240502 07:59:24.652056 22307 consensus_peers.cc:485] T b2abf00f982e450fbe0124226789324a P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:24.652176 1872 tablet_replica.cc:330] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014: stopping tablet replica
I20240502 07:59:24.652308 1872 raft_consensus.cc:2229] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:24.652417 1872 raft_consensus.cc:2258] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:24.652921 1872 ts_tablet_manager.cc:1875] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:24.653244 1835 ts_tablet_manager.cc:1888] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:24.653302 1835 log.cc:1196] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/dac19fc2557248cc900081045ecea2a9
I20240502 07:59:24.653579 1835 ts_tablet_manager.cc:1909] T dac19fc2557248cc900081045ecea2a9 P 1aecef81733c47b3b468504934357231: Deleting consensus metadata
I20240502 07:59:24.654764 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet dac19fc2557248cc900081045ecea2a9 (table rebalancer_dynamic_table_new_name [id=142b302e127f40e4808828e001274c13]) successfully deleted
W20240502 07:59:24.655704 22569 consensus_peers.cc:485] T 5ca68d8eec874332b74df02eb55e0ff9 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:24.656044 1873 ts_tablet_manager.cc:1888] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:24.656476 1873 log.cc:1196] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/dac19fc2557248cc900081045ecea2a9
I20240502 07:59:24.656471 1872 ts_tablet_manager.cc:1888] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:24.656538 1872 log.cc:1196] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/wals/dac19fc2557248cc900081045ecea2a9
I20240502 07:59:24.656896 1872 ts_tablet_manager.cc:1909] T dac19fc2557248cc900081045ecea2a9 P f00f3a1876444ad2bf9c919068ab8014: Deleting consensus metadata
I20240502 07:59:24.657023 1873 ts_tablet_manager.cc:1909] T dac19fc2557248cc900081045ecea2a9 P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:59:24.659924 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet dac19fc2557248cc900081045ecea2a9 (table rebalancer_dynamic_table_new_name [id=142b302e127f40e4808828e001274c13]) successfully deleted
I20240502 07:59:24.660137 22217 catalog_manager.cc:4830] TS f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673): tablet dac19fc2557248cc900081045ecea2a9 (table rebalancer_dynamic_table_new_name [id=142b302e127f40e4808828e001274c13]) successfully deleted
W20240502 07:59:24.738443 22569 consensus_peers.cc:485] T 855b2d71ec18485dbd43acaf4fc1d986 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:24.746837 1615 raft_consensus.cc:482] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:24.746977 1615 raft_consensus.cc:504] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:24.747417 1615 leader_election.cc:288] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:24.747920 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2e55f41eb174341bea30ccee0f50e15" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014" is_pre_election: true
I20240502 07:59:24.748056 22763 raft_consensus.cc:2454] T b2e55f41eb174341bea30ccee0f50e15 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:24.748072 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2e55f41eb174341bea30ccee0f50e15" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c" is_pre_election: true
I20240502 07:59:24.748209 22501 raft_consensus.cc:2454] T b2e55f41eb174341bea30ccee0f50e15 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 0.
I20240502 07:59:24.748499 22567 leader_election.cc:302] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:24.748709 1615 raft_consensus.cc:2781] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:24.748759 1615 raft_consensus.cc:482] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:24.748804 1615 raft_consensus.cc:3037] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.749876 1615 raft_consensus.cc:504] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:24.750231 1615 leader_election.cc:288] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Requested vote from peers f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132:41673), fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625)
I20240502 07:59:24.750655 22763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2e55f41eb174341bea30ccee0f50e15" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00f3a1876444ad2bf9c919068ab8014"
I20240502 07:59:24.750772 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2e55f41eb174341bea30ccee0f50e15" candidate_uuid: "e51b10af49484aaa8f0267aebc1bf27f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fabddb16c85944648a710ec52fc7244c"
I20240502 07:59:24.750809 22763 raft_consensus.cc:3037] T b2e55f41eb174341bea30ccee0f50e15 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.750880 22501 raft_consensus.cc:3037] T b2e55f41eb174341bea30ccee0f50e15 P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.751839 22501 raft_consensus.cc:2454] T b2e55f41eb174341bea30ccee0f50e15 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:24.751839 22763 raft_consensus.cc:2454] T b2e55f41eb174341bea30ccee0f50e15 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51b10af49484aaa8f0267aebc1bf27f in term 1.
I20240502 07:59:24.752142 22570 leader_election.cc:302] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e51b10af49484aaa8f0267aebc1bf27f, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:24.752314 1615 raft_consensus.cc:2781] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:24.752400 1615 raft_consensus.cc:686] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Becoming Leader. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Running, Role: LEADER
I20240502 07:59:24.752542 1615 consensus_queue.cc:237] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:24.755369 22223 catalog_manager.cc:5484] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f reported cstate change: term changed from 0 to 1, leader changed from <none> to e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131). New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:24.756927 22224 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:24.762483 24930 tablet_service.cc:1467] Processing CreateTablet for tablet ef620cd64d8d4ff8b565b99d3065c04c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=64a0059a39da4fb497b51a56fbb6f16f]), partition=
I20240502 07:59:24.762773 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef620cd64d8d4ff8b565b99d3065c04c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.763118 22613 tablet_service.cc:1467] Processing CreateTablet for tablet ef620cd64d8d4ff8b565b99d3065c04c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=64a0059a39da4fb497b51a56fbb6f16f]), partition=
I20240502 07:59:24.763324 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef620cd64d8d4ff8b565b99d3065c04c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.765347 22351 tablet_service.cc:1467] Processing CreateTablet for tablet ef620cd64d8d4ff8b565b99d3065c04c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=64a0059a39da4fb497b51a56fbb6f16f]), partition=
I20240502 07:59:24.765583 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef620cd64d8d4ff8b565b99d3065c04c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:24.766023 1801 tablet_bootstrap.cc:492] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:24.766963 1801 tablet_bootstrap.cc:654] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.767800 1407 tablet_bootstrap.cc:492] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:24.768245 1801 tablet_bootstrap.cc:492] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:24.768324 1801 ts_tablet_manager.cc:1367] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:24.768540 1511 tablet_bootstrap.cc:492] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:24.768762 1407 tablet_bootstrap.cc:654] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.768874 1801 raft_consensus.cc:348] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:24.769016 1801 raft_consensus.cc:374] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.769057 1801 raft_consensus.cc:729] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.769182 1801 consensus_queue.cc:260] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:24.769583 1801 ts_tablet_manager.cc:1398] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:24.770759 1407 tablet_bootstrap.cc:492] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:24.770843 1407 ts_tablet_manager.cc:1367] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:24.771286 1407 raft_consensus.cc:348] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:24.771394 1407 raft_consensus.cc:374] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.771421 1407 raft_consensus.cc:729] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.771538 1407 consensus_queue.cc:260] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:24.771808 1407 ts_tablet_manager.cc:1398] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240502 07:59:24.772722 24887 consensus_peers.cc:485] T 7c09dfefb9c246c684884973bdc35c75 P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:24.781697 1511 tablet_bootstrap.cc:654] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:24.784132 1511 tablet_bootstrap.cc:492] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:24.784240 1511 ts_tablet_manager.cc:1367] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20240502 07:59:24.784788 1511 raft_consensus.cc:348] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:24.784886 1511 raft_consensus.cc:374] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:24.784906 1511 raft_consensus.cc:729] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:24.785017 1511 consensus_queue.cc:260] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:24.785362 1511 ts_tablet_manager.cc:1398] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:24.787731 25193 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:24.797768 25063 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:24.800843 22613 tablet_service.cc:1429] Tablet server has 28 leaders and 0 scanners
I20240502 07:59:24.806860 24930 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:24.807651 22482 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:24.818090 1740 raft_consensus.cc:482] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:24.818387 1740 raft_consensus.cc:504] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:24.818986 1740 leader_election.cc:288] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:24.820058 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef620cd64d8d4ff8b565b99d3065c04c" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:24.820197 22634 raft_consensus.cc:2454] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:24.820520 22308 leader_election.cc:302] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:24.820710 1740 raft_consensus.cc:2781] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:24.820775 1740 raft_consensus.cc:482] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:24.820824 1740 raft_consensus.cc:3037] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.820784 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef620cd64d8d4ff8b565b99d3065c04c" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:24.820907 24951 raft_consensus.cc:2454] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:24.821957 1740 raft_consensus.cc:504] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:24.822288 1740 leader_election.cc:288] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:24.822697 24952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef620cd64d8d4ff8b565b99d3065c04c" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:24.822702 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef620cd64d8d4ff8b565b99d3065c04c" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:24.822803 24952 raft_consensus.cc:3037] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.822816 22634 raft_consensus.cc:3037] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:24.823673 22634 raft_consensus.cc:2454] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:24.823797 24952 raft_consensus.cc:2454] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:24.823946 22308 leader_election.cc:302] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:24.824090 1740 raft_consensus.cc:2781] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:24.824153 1740 raft_consensus.cc:686] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:59:24.824273 1740 consensus_queue.cc:237] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:24.824515 22351 tablet_service.cc:1429] Tablet server has 16 leaders and 0 scanners
I20240502 07:59:24.827229 22745 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:24.826911 22223 catalog_manager.cc:5484] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
W20240502 07:59:24.907874 25019 consensus_peers.cc:485] T bf443ada59854e6da532aa5832a9a564 P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:24.921168 22700 consensus_peers.cc:462] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:24.944903 22227 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "a" } } modify_external_catalogs: true
I20240502 07:59:24.948853 22634 raft_consensus.cc:1261] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:24.948853 24952 raft_consensus.cc:1261] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:24.949541 1740 consensus_queue.cc:1035] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:24.949694 32390 consensus_queue.cc:1035] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:24.954425 22225 catalog_manager.cc:5576] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703) does not have the latest schema for tablet ef620cd64d8d4ff8b565b99d3065c04c (table rebalancer_dynamic_table [id=64a0059a39da4fb497b51a56fbb6f16f]). Expected version 1 got 0
I20240502 07:59:24.957093 22228 catalog_manager.cc:5576] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995) does not have the latest schema for tablet ef620cd64d8d4ff8b565b99d3065c04c (table rebalancer_dynamic_table [id=64a0059a39da4fb497b51a56fbb6f16f]). Expected version 1 got 0
W20240502 07:59:24.957520 22438 consensus_peers.cc:485] T d61cb2b6dce1409bb30d1c68c970dc22 P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:24.962671 1740 consensus_queue.cc:1035] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:24.967708 1582 tablet.cc:1722] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:24.968137 1582 tablet.cc:1620] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:24.969877 1583 tablet.cc:1722] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:24.970265 1583 tablet.cc:1620] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:24.977727 1673 consensus_queue.cc:1035] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:24.980594 32390 consensus_queue.cc:1035] T c3a4b31595124c2681d6430c4748d3ff P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:24.985808 22221 catalog_manager.cc:5576] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703) does not have the latest schema for tablet ef620cd64d8d4ff8b565b99d3065c04c (table rebalancer_dynamic_table [id=64a0059a39da4fb497b51a56fbb6f16f]). Expected version 1 got 0
I20240502 07:59:24.986851 22222 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=64a0059a39da4fb497b51a56fbb6f16f] alter complete (version 1)
W20240502 07:59:24.989780 22569 consensus_peers.cc:462] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:25.000828 1935 tablet.cc:1722] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
11:a INT64 NULLABLE,
PRIMARY KEY (key)
) version 0 to (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1
I20240502 07:59:25.003079 1935 tablet.cc:1620] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
I20240502 07:59:25.005587 601 consensus_queue.cc:1035] T cdc3697b2b8f4306ae0901b650e334b3 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:25.027750 22613 tablet_service.cc:1429] Tablet server has 28 leaders and 0 scanners
I20240502 07:59:25.029407 24930 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:25.032866 25063 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:25.033474 25193 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
W20240502 07:59:25.042852 25150 consensus_peers.cc:462] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:25.049187 22351 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:25.049497 22482 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:25.061897 22745 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
W20240502 07:59:25.088263 25150 consensus_peers.cc:485] T e3fd481f3c78497eb629b0180c07674f P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.095032 22223 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "a" type: STRING is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20240502 07:59:25.099922 1582 tablet.cc:1722] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:25.100246 1582 tablet.cc:1620] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4: MemRowSet was empty: no flush needed.
I20240502 07:59:25.100533 1935 tablet.cc:1722] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:25.100836 1935 tablet.cc:1620] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f: MemRowSet was empty: no flush needed.
I20240502 07:59:25.101086 1583 tablet.cc:1722] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2
I20240502 07:59:25.101420 1583 tablet.cc:1620] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231: MemRowSet was empty: no flush needed.
I20240502 07:59:25.106504 22223 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table [id=64a0059a39da4fb497b51a56fbb6f16f] alter complete (version 2)
W20240502 07:59:25.132735 25019 consensus_peers.cc:485] T 4423a78b96cd42f8a509dc1f28d87a7f P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.141499 1933 consensus_queue.cc:1035] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 07:59:25.145992 22569 consensus_peers.cc:485] T f033e8bbac8445f3af64faa67c818987 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.146654 1933 consensus_queue.cc:1035] T b2e55f41eb174341bea30ccee0f50e15 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 07:59:25.154784 24887 consensus_peers.cc:485] T 4840fa82fcb241f391c4890fb735fe9a P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:25.203344 25150 consensus_peers.cc:485] T b27345a5a0874524855f57f8903afe6c P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.209159 22228 catalog_manager.cc:3330] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table" } new_table_name: "rebalancer_dynamic_table_new_name" modify_external_catalogs: true
W20240502 07:59:25.213207 25150 consensus_peers.cc:485] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:25.218448 24887 consensus_peers.cc:485] T caf2dd9c96b94566b216692a9370f16e P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.224622 1582 tablet.cc:1722] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:25.225165 1935 tablet.cc:1722] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
I20240502 07:59:25.236379 22228 catalog_manager.cc:6029] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: rebalancer_dynamic_table_new_name [id=64a0059a39da4fb497b51a56fbb6f16f] alter complete (version 3)
I20240502 07:59:25.238965 1583 tablet.cc:1722] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231: Alter schema from (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 2 to (
10:key INT64 NOT NULL,
12:a STRING NULLABLE,
PRIMARY KEY (key)
) version 3
W20240502 07:59:25.242142 22569 consensus_peers.cc:485] T 71d8cf6286aa4c3cb4f87d5974cf8548 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:25.245851 22438 consensus_peers.cc:485] T 80b043e06deb4582b06d1e0e5d8a0658 P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20240502 07:59:25.264025 25193 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
W20240502 07:59:25.268112 24887 consensus_peers.cc:485] T cba71d2a28984e84b1950d59ec2e5a6a P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.274356 22351 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
W20240502 07:59:25.275900 22216 catalog_manager.cc:4573] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8ef453d0a2eb4873b25903242ca39a96: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:25.276193 22482 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:25.281265 22228 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_extra_table_70"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:25.283475 24930 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:25.290653 25063 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:25.296723 22351 tablet_service.cc:1467] Processing CreateTablet for tablet 4d06f144440146fcb7cfe97cdd6fbff8 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=d4a43029a4f341ef8ef5e06bdba0ba24]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:25.297035 22351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d06f144440146fcb7cfe97cdd6fbff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:25.298048 22352 tablet_service.cc:1467] Processing CreateTablet for tablet 317b0e49db674cb2bec93398c830bc85 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=d4a43029a4f341ef8ef5e06bdba0ba24]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:25.298288 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 317b0e49db674cb2bec93398c830bc85. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:25.299544 22613 tablet_service.cc:1467] Processing CreateTablet for tablet 4d06f144440146fcb7cfe97cdd6fbff8 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=d4a43029a4f341ef8ef5e06bdba0ba24]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:25.299787 22613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d06f144440146fcb7cfe97cdd6fbff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:25.299834 2007 tablet_bootstrap.cc:492] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:25.300443 22612 tablet_service.cc:1467] Processing CreateTablet for tablet 317b0e49db674cb2bec93398c830bc85 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=d4a43029a4f341ef8ef5e06bdba0ba24]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:25.300674 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 317b0e49db674cb2bec93398c830bc85. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:25.300748 2007 tablet_bootstrap.cc:654] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:25.301949 2009 tablet_bootstrap.cc:492] T 4d06f144440146fcb7cfe97cdd6fbff8 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:25.302079 22614 tablet_service.cc:1467] Processing CreateTablet for tablet 9be2b30a7d80488db57445063ee35020 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=d4a43029a4f341ef8ef5e06bdba0ba24]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:25.302294 22614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9be2b30a7d80488db57445063ee35020. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:25.302786 22745 tablet_service.cc:1467] Processing CreateTablet for tablet 9be2b30a7d80488db57445063ee35020 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=d4a43029a4f341ef8ef5e06bdba0ba24]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:25.302858 2009 tablet_bootstrap.cc:654] T 4d06f144440146fcb7cfe97cdd6fbff8 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:25.303030 22745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9be2b30a7d80488db57445063ee35020. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:25.304384 24931 tablet_service.cc:1467] Processing CreateTablet for tablet 317b0e49db674cb2bec93398c830bc85 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=d4a43029a4f341ef8ef5e06bdba0ba24]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:25.304618 24931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 317b0e49db674cb2bec93398c830bc85. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:25.305298 2011 tablet_bootstrap.cc:492] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014: Bootstrap starting.
I20240502 07:59:25.305974 2011 tablet_bootstrap.cc:654] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:25.306736 2013 tablet_bootstrap.cc:492] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:25.307849 2013 tablet_bootstrap.cc:654] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:25.309576 24930 tablet_service.cc:1467] Processing CreateTablet for tablet 4d06f144440146fcb7cfe97cdd6fbff8 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=d4a43029a4f341ef8ef5e06bdba0ba24]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:25.309825 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d06f144440146fcb7cfe97cdd6fbff8. 1 dirs total, 0 dirs full, 0 dirs failed
W20240502 07:59:25.311455 22438 consensus_peers.cc:485] T a0430566f5ae4ffabcf15bb8b8af6bb5 P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:25.311885 22307 consensus_peers.cc:485] T 02b326ae3ffa4afbb7a1dcfbad4c56ba P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.312405 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 9be2b30a7d80488db57445063ee35020 (DEFAULT_TABLE table=rebalancer_extra_table_70 [id=d4a43029a4f341ef8ef5e06bdba0ba24]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:25.312673 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9be2b30a7d80488db57445063ee35020. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:25.314843 2015 tablet_bootstrap.cc:492] T 9be2b30a7d80488db57445063ee35020 P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
I20240502 07:59:25.316053 2015 tablet_bootstrap.cc:654] T 9be2b30a7d80488db57445063ee35020 P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:25.320103 2007 tablet_bootstrap.cc:492] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:25.320211 2007 ts_tablet_manager.cc:1367] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20240502 07:59:25.320653 2007 raft_consensus.cc:348] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:25.320755 2007 raft_consensus.cc:374] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:25.320780 2007 raft_consensus.cc:729] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:25.320894 2007 consensus_queue.cc:260] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:25.321182 2007 ts_tablet_manager.cc:1398] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:25.321501 2007 tablet_bootstrap.cc:492] T 317b0e49db674cb2bec93398c830bc85 P b533a7d8d97943609c6e28e3e36b78a4: Bootstrap starting.
I20240502 07:59:25.322289 2007 tablet_bootstrap.cc:654] T 317b0e49db674cb2bec93398c830bc85 P b533a7d8d97943609c6e28e3e36b78a4: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:25.323382 2007 tablet_bootstrap.cc:492] T 317b0e49db674cb2bec93398c830bc85 P b533a7d8d97943609c6e28e3e36b78a4: No bootstrap required, opened a new log
I20240502 07:59:25.323522 2007 ts_tablet_manager.cc:1367] T 317b0e49db674cb2bec93398c830bc85 P b533a7d8d97943609c6e28e3e36b78a4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:25.324096 2007 raft_consensus.cc:348] T 317b0e49db674cb2bec93398c830bc85 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:25.324262 2007 raft_consensus.cc:374] T 317b0e49db674cb2bec93398c830bc85 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:25.324347 2007 raft_consensus.cc:729] T 317b0e49db674cb2bec93398c830bc85 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Initialized, Role: FOLLOWER
I20240502 07:59:25.324537 2007 consensus_queue.cc:260] T 317b0e49db674cb2bec93398c830bc85 P b533a7d8d97943609c6e28e3e36b78a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:25.324940 2007 ts_tablet_manager.cc:1398] T 317b0e49db674cb2bec93398c830bc85 P b533a7d8d97943609c6e28e3e36b78a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:25.326380 2011 tablet_bootstrap.cc:492] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014: No bootstrap required, opened a new log
I20240502 07:59:25.326485 2011 ts_tablet_manager.cc:1367] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.001s
W20240502 07:59:25.326658 22307 consensus_peers.cc:485] T f5f0690b92c140c49fe10ff46e0136df P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.327638 2011 raft_consensus.cc:348] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:25.329044 2011 raft_consensus.cc:374] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:25.329142 2011 raft_consensus.cc:729] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Initialized, Role: FOLLOWER
I20240502 07:59:25.329345 2011 consensus_queue.cc:260] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:25.331434 22745 tablet_service.cc:1429] Tablet server has 14 leaders and 0 scanners
I20240502 07:59:25.332679 2015 tablet_bootstrap.cc:492] T 9be2b30a7d80488db57445063ee35020 P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:25.332772 2015 ts_tablet_manager.cc:1367] T 9be2b30a7d80488db57445063ee35020 P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20240502 07:59:25.333134 2011 ts_tablet_manager.cc:1398] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20240502 07:59:25.333276 2015 raft_consensus.cc:348] T 9be2b30a7d80488db57445063ee35020 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:25.333393 2015 raft_consensus.cc:374] T 9be2b30a7d80488db57445063ee35020 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:25.333420 2015 raft_consensus.cc:729] T 9be2b30a7d80488db57445063ee35020 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:25.333523 2015 consensus_queue.cc:260] T 9be2b30a7d80488db57445063ee35020 P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:25.333784 2015 ts_tablet_manager.cc:1398] T 9be2b30a7d80488db57445063ee35020 P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240502 07:59:25.338264 24887 consensus_peers.cc:485] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Error code: TABLET_NOT_FOUND (6). Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.341811 22228 catalog_manager.cc:2458] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:25.341941 22228 catalog_manager.cc:2642] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47042:
table { table_name: "rebalancer_dynamic_table_new_name" } modify_external_catalogs: true
I20240502 07:59:25.344255 22228 catalog_manager.cc:5771] T 00000000000000000000000000000000 P f9724f21fce0432bad8bbeb45d297445: Sending DeleteTablet for 3 replicas of tablet ef620cd64d8d4ff8b565b99d3065c04c
I20240502 07:59:25.344740 22352 tablet_service.cc:1514] Processing DeleteTablet for tablet ef620cd64d8d4ff8b565b99d3065c04c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:25 UTC) from {username='slave'} at 127.0.0.1:43702
I20240502 07:59:25.344959 22614 tablet_service.cc:1514] Processing DeleteTablet for tablet ef620cd64d8d4ff8b565b99d3065c04c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:25 UTC) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:25.345256 2019 tablet_replica.cc:330] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:25.345484 2019 raft_consensus.cc:2229] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:25.345728 24930 tablet_service.cc:1514] Processing DeleteTablet for tablet ef620cd64d8d4ff8b565b99d3065c04c with delete_type TABLET_DATA_DELETED (Table deleted at 2024-05-02 07:59:25 UTC) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:25.345988 2019 raft_consensus.cc:2258] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:25.346942 2019 ts_tablet_manager.cc:1875] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:25.349066 2019 ts_tablet_manager.cc:1888] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:25.349138 2019 log.cc:1196] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/ef620cd64d8d4ff8b565b99d3065c04c
I20240502 07:59:25.349488 2019 ts_tablet_manager.cc:1909] T ef620cd64d8d4ff8b565b99d3065c04c P e51b10af49484aaa8f0267aebc1bf27f: Deleting consensus metadata
I20240502 07:59:25.350838 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet ef620cd64d8d4ff8b565b99d3065c04c (table rebalancer_dynamic_table_new_name [id=64a0059a39da4fb497b51a56fbb6f16f]) successfully deleted
I20240502 07:59:25.353508 2020 tablet_replica.cc:330] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:25.353657 2020 raft_consensus.cc:2229] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:25.353777 2020 raft_consensus.cc:2258] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:25.354396 2020 ts_tablet_manager.cc:1875] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:25.354424 2018 tablet_replica.cc:330] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4: stopping tablet replica
I20240502 07:59:25.354568 2018 raft_consensus.cc:2229] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Raft consensus shutting down.
I20240502 07:59:25.355037 2018 raft_consensus.cc:2258] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:25.356197 2018 ts_tablet_manager.cc:1875] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240502 07:59:25.358749 2018 ts_tablet_manager.cc:1888] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:25.358824 2018 log.cc:1196] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/wals/ef620cd64d8d4ff8b565b99d3065c04c
I20240502 07:59:25.359479 2018 ts_tablet_manager.cc:1909] T ef620cd64d8d4ff8b565b99d3065c04c P b533a7d8d97943609c6e28e3e36b78a4: Deleting consensus metadata
I20240502 07:59:25.359810 2009 tablet_bootstrap.cc:492] T 4d06f144440146fcb7cfe97cdd6fbff8 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:25.359810 2013 tablet_bootstrap.cc:492] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:25.359900 2013 ts_tablet_manager.cc:1367] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.053s user 0.002s sys 0.000s
I20240502 07:59:25.359900 2009 ts_tablet_manager.cc:1367] T 4d06f144440146fcb7cfe97cdd6fbff8 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.058s user 0.001s sys 0.000s
I20240502 07:59:25.360343 22217 catalog_manager.cc:4830] TS b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): tablet ef620cd64d8d4ff8b565b99d3065c04c (table rebalancer_dynamic_table_new_name [id=64a0059a39da4fb497b51a56fbb6f16f]) successfully deleted
I20240502 07:59:25.360356 2009 raft_consensus.cc:348] T 4d06f144440146fcb7cfe97cdd6fbff8 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:25.360356 2013 raft_consensus.cc:348] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:25.360472 2009 raft_consensus.cc:374] T 4d06f144440146fcb7cfe97cdd6fbff8 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:25.360472 2013 raft_consensus.cc:374] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:25.360505 2013 raft_consensus.cc:729] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:25.360505 2009 raft_consensus.cc:729] T 4d06f144440146fcb7cfe97cdd6fbff8 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:25.360616 2013 consensus_queue.cc:260] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:25.360616 2009 consensus_queue.cc:260] T 4d06f144440146fcb7cfe97cdd6fbff8 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:25.360889 2009 ts_tablet_manager.cc:1398] T 4d06f144440146fcb7cfe97cdd6fbff8 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:25.360889 2013 ts_tablet_manager.cc:1398] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:25.361042 2009 tablet_bootstrap.cc:492] T 317b0e49db674cb2bec93398c830bc85 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:25.361042 2013 tablet_bootstrap.cc:492] T 4d06f144440146fcb7cfe97cdd6fbff8 P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:25.361984 2009 tablet_bootstrap.cc:654] T 317b0e49db674cb2bec93398c830bc85 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:25.362327 2013 tablet_bootstrap.cc:654] T 4d06f144440146fcb7cfe97cdd6fbff8 P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
W20240502 07:59:25.363885 22569 consensus_peers.cc:485] T b270d89546214f4d9d15193f4f8e04d5 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:25.365253 25150 consensus_peers.cc:485] T 2ea24049cec547b0a6cb85be6033eb3d P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.366307 22614 tablet_service.cc:1429] Tablet server has 28 leaders and 0 scanners
I20240502 07:59:25.366636 2009 tablet_bootstrap.cc:492] T 317b0e49db674cb2bec93398c830bc85 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:25.366724 2009 ts_tablet_manager.cc:1367] T 317b0e49db674cb2bec93398c830bc85 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:25.367158 2009 raft_consensus.cc:348] T 317b0e49db674cb2bec93398c830bc85 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:25.367270 2009 raft_consensus.cc:374] T 317b0e49db674cb2bec93398c830bc85 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:25.367311 2009 raft_consensus.cc:729] T 317b0e49db674cb2bec93398c830bc85 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:25.367439 2009 consensus_queue.cc:260] T 317b0e49db674cb2bec93398c830bc85 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:25.367740 2009 ts_tablet_manager.cc:1398] T 317b0e49db674cb2bec93398c830bc85 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:25.368155 2009 tablet_bootstrap.cc:492] T 9be2b30a7d80488db57445063ee35020 P e51b10af49484aaa8f0267aebc1bf27f: Bootstrap starting.
I20240502 07:59:25.368638 2020 ts_tablet_manager.cc:1888] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20240502 07:59:25.368737 2020 log.cc:1196] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/ef620cd64d8d4ff8b565b99d3065c04c
I20240502 07:59:25.369268 2009 tablet_bootstrap.cc:654] T 9be2b30a7d80488db57445063ee35020 P e51b10af49484aaa8f0267aebc1bf27f: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:25.369552 2020 ts_tablet_manager.cc:1909] T ef620cd64d8d4ff8b565b99d3065c04c P 1aecef81733c47b3b468504934357231: Deleting consensus metadata
I20240502 07:59:25.371209 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet ef620cd64d8d4ff8b565b99d3065c04c (table rebalancer_dynamic_table_new_name [id=64a0059a39da4fb497b51a56fbb6f16f]) successfully deleted
I20240502 07:59:25.372761 2009 tablet_bootstrap.cc:492] T 9be2b30a7d80488db57445063ee35020 P e51b10af49484aaa8f0267aebc1bf27f: No bootstrap required, opened a new log
I20240502 07:59:25.372835 2009 ts_tablet_manager.cc:1367] T 9be2b30a7d80488db57445063ee35020 P e51b10af49484aaa8f0267aebc1bf27f: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:25.373236 2009 raft_consensus.cc:348] T 9be2b30a7d80488db57445063ee35020 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:25.373369 2009 raft_consensus.cc:374] T 9be2b30a7d80488db57445063ee35020 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:25.373525 2009 raft_consensus.cc:729] T 9be2b30a7d80488db57445063ee35020 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51b10af49484aaa8f0267aebc1bf27f, State: Initialized, Role: FOLLOWER
I20240502 07:59:25.373654 2009 consensus_queue.cc:260] T 9be2b30a7d80488db57445063ee35020 P e51b10af49484aaa8f0267aebc1bf27f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:25.373930 2009 ts_tablet_manager.cc:1398] T 9be2b30a7d80488db57445063ee35020 P e51b10af49484aaa8f0267aebc1bf27f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:25.374316 2013 tablet_bootstrap.cc:492] T 4d06f144440146fcb7cfe97cdd6fbff8 P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:25.374420 2013 ts_tablet_manager.cc:1367] T 4d06f144440146fcb7cfe97cdd6fbff8 P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20240502 07:59:25.374887 2013 raft_consensus.cc:348] T 4d06f144440146fcb7cfe97cdd6fbff8 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:25.374998 2013 raft_consensus.cc:374] T 4d06f144440146fcb7cfe97cdd6fbff8 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:25.375036 2013 raft_consensus.cc:729] T 4d06f144440146fcb7cfe97cdd6fbff8 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:25.375177 2013 consensus_queue.cc:260] T 4d06f144440146fcb7cfe97cdd6fbff8 P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:25.375458 2013 ts_tablet_manager.cc:1398] T 4d06f144440146fcb7cfe97cdd6fbff8 P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240502 07:59:25.380556 22700 consensus_peers.cc:462] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:25.395896 31905 raft_consensus.cc:482] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:25.396026 31905 raft_consensus.cc:504] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:25.396560 31905 leader_election.cc:288] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
I20240502 07:59:25.396816 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9be2b30a7d80488db57445063ee35020" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:25.396929 25083 raft_consensus.cc:2454] T 9be2b30a7d80488db57445063ee35020 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:25.397259 22698 leader_election.cc:302] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:25.397625 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9be2b30a7d80488db57445063ee35020" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:25.397756 22634 raft_consensus.cc:2454] T 9be2b30a7d80488db57445063ee35020 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 0.
I20240502 07:59:25.397807 31905 raft_consensus.cc:2781] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:25.397862 31905 raft_consensus.cc:482] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:25.397900 31905 raft_consensus.cc:3037] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:25.398962 31905 raft_consensus.cc:504] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:25.399387 31905 leader_election.cc:288] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Requested vote from peers 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703)
W20240502 07:59:25.399571 25019 consensus_peers.cc:485] T e75b8171bf164f748fce5a71d86c41bf P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.400974 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9be2b30a7d80488db57445063ee35020" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:25.400987 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9be2b30a7d80488db57445063ee35020" candidate_uuid: "f00f3a1876444ad2bf9c919068ab8014" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:25.401085 22634 raft_consensus.cc:3037] T 9be2b30a7d80488db57445063ee35020 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:25.401154 25083 raft_consensus.cc:3037] T 9be2b30a7d80488db57445063ee35020 P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:25.401894 22634 raft_consensus.cc:2454] T 9be2b30a7d80488db57445063ee35020 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:25.401898 25083 raft_consensus.cc:2454] T 9be2b30a7d80488db57445063ee35020 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f00f3a1876444ad2bf9c919068ab8014 in term 1.
I20240502 07:59:25.402201 22698 leader_election.cc:302] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, f00f3a1876444ad2bf9c919068ab8014; no voters:
I20240502 07:59:25.402735 31905 raft_consensus.cc:2781] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:25.402824 31905 raft_consensus.cc:686] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Becoming Leader. State: Replica: f00f3a1876444ad2bf9c919068ab8014, State: Running, Role: LEADER
I20240502 07:59:25.402958 31905 consensus_queue.cc:237] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:25.405597 22224 catalog_manager.cc:5484] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: term changed from 0 to 1, leader changed from <none> to f00f3a1876444ad2bf9c919068ab8014 (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
W20240502 07:59:25.406271 22307 consensus_peers.cc:485] T f827f414e7cc4ede8e41716b6e2efcf9 P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:25.423789 22569 consensus_peers.cc:485] T a0791c02721040e495462142c7bc9e35 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.447224 22224 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47042:
name: "rebalancer_dynamic_table"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "a"
type: INT64
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
I20240502 07:59:25.454381 22482 tablet_service.cc:1467] Processing CreateTablet for tablet e0e03a628f9b4119936b884680202c0c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9a4eb3f45d304031ae72ca92e6275bd0]), partition=
I20240502 07:59:25.454676 22482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0e03a628f9b4119936b884680202c0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:25.454708 24930 tablet_service.cc:1467] Processing CreateTablet for tablet e0e03a628f9b4119936b884680202c0c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9a4eb3f45d304031ae72ca92e6275bd0]), partition=
I20240502 07:59:25.454938 24930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0e03a628f9b4119936b884680202c0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:25.457053 2066 tablet_bootstrap.cc:492] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c: Bootstrap starting.
I20240502 07:59:25.457013 2013 tablet_bootstrap.cc:492] T e0e03a628f9b4119936b884680202c0c P 1aecef81733c47b3b468504934357231: Bootstrap starting.
I20240502 07:59:25.457232 25063 tablet_service.cc:1467] Processing CreateTablet for tablet e0e03a628f9b4119936b884680202c0c (DEFAULT_TABLE table=rebalancer_dynamic_table [id=9a4eb3f45d304031ae72ca92e6275bd0]), partition=
I20240502 07:59:25.457477 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0e03a628f9b4119936b884680202c0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:25.458200 2013 tablet_bootstrap.cc:654] T e0e03a628f9b4119936b884680202c0c P 1aecef81733c47b3b468504934357231: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:25.458653 2066 tablet_bootstrap.cc:654] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:25.459978 2015 tablet_bootstrap.cc:492] T e0e03a628f9b4119936b884680202c0c P 227eec4b5f014d349661f6e0b39b9055: Bootstrap starting.
W20240502 07:59:25.460835 25019 consensus_peers.cc:485] T a9c05f7717854c85a316eb76f9965f23 P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.460940 2015 tablet_bootstrap.cc:654] T e0e03a628f9b4119936b884680202c0c P 227eec4b5f014d349661f6e0b39b9055: Neither blocks nor log segments found. Creating new log.
W20240502 07:59:25.461134 22438 consensus_peers.cc:485] T 0695ee734e4d494ca64898920ffe9566 P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.462960 2013 tablet_bootstrap.cc:492] T e0e03a628f9b4119936b884680202c0c P 1aecef81733c47b3b468504934357231: No bootstrap required, opened a new log
I20240502 07:59:25.463052 2013 ts_tablet_manager.cc:1367] T e0e03a628f9b4119936b884680202c0c P 1aecef81733c47b3b468504934357231: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:25.463518 2013 raft_consensus.cc:348] T e0e03a628f9b4119936b884680202c0c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:25.463625 2013 raft_consensus.cc:374] T e0e03a628f9b4119936b884680202c0c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:25.463660 2013 raft_consensus.cc:729] T e0e03a628f9b4119936b884680202c0c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aecef81733c47b3b468504934357231, State: Initialized, Role: FOLLOWER
I20240502 07:59:25.463774 2013 consensus_queue.cc:260] T e0e03a628f9b4119936b884680202c0c P 1aecef81733c47b3b468504934357231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:25.464282 2013 ts_tablet_manager.cc:1398] T e0e03a628f9b4119936b884680202c0c P 1aecef81733c47b3b468504934357231: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:25.464990 2015 tablet_bootstrap.cc:492] T e0e03a628f9b4119936b884680202c0c P 227eec4b5f014d349661f6e0b39b9055: No bootstrap required, opened a new log
I20240502 07:59:25.465076 2015 ts_tablet_manager.cc:1367] T e0e03a628f9b4119936b884680202c0c P 227eec4b5f014d349661f6e0b39b9055: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:25.465556 2015 raft_consensus.cc:348] T e0e03a628f9b4119936b884680202c0c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:25.465662 2015 raft_consensus.cc:374] T e0e03a628f9b4119936b884680202c0c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:25.465698 2015 raft_consensus.cc:729] T e0e03a628f9b4119936b884680202c0c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227eec4b5f014d349661f6e0b39b9055, State: Initialized, Role: FOLLOWER
I20240502 07:59:25.465822 2015 consensus_queue.cc:260] T e0e03a628f9b4119936b884680202c0c P 227eec4b5f014d349661f6e0b39b9055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:25.466125 2015 ts_tablet_manager.cc:1398] T e0e03a628f9b4119936b884680202c0c P 227eec4b5f014d349661f6e0b39b9055: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:25.468906 308 raft_consensus.cc:482] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:25.469002 308 raft_consensus.cc:504] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:25.469367 308 leader_election.cc:288] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
W20240502 07:59:25.469767 22569 consensus_peers.cc:485] T 8477b961ce384edea078851122e196e9 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.470109 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d06f144440146fcb7cfe97cdd6fbff8" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:25.470243 22634 raft_consensus.cc:2454] T 4d06f144440146fcb7cfe97cdd6fbff8 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:25.470536 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d06f144440146fcb7cfe97cdd6fbff8" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:25.470655 24951 raft_consensus.cc:2454] T 4d06f144440146fcb7cfe97cdd6fbff8 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 0.
I20240502 07:59:25.471311 22308 leader_election.cc:302] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b533a7d8d97943609c6e28e3e36b78a4, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:25.471485 308 raft_consensus.cc:2781] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:25.471545 308 raft_consensus.cc:482] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:25.471583 308 raft_consensus.cc:3037] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:25.472702 308 raft_consensus.cc:504] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:25.473026 308 leader_election.cc:288] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), 1aecef81733c47b3b468504934357231 (127.15.129.134:35995)
I20240502 07:59:25.474123 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d06f144440146fcb7cfe97cdd6fbff8" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:25.474260 24951 raft_consensus.cc:3037] T 4d06f144440146fcb7cfe97cdd6fbff8 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:25.474807 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d06f144440146fcb7cfe97cdd6fbff8" candidate_uuid: "b533a7d8d97943609c6e28e3e36b78a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:25.474927 22634 raft_consensus.cc:3037] T 4d06f144440146fcb7cfe97cdd6fbff8 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:25.475306 24951 raft_consensus.cc:2454] T 4d06f144440146fcb7cfe97cdd6fbff8 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:25.475911 22634 raft_consensus.cc:2454] T 4d06f144440146fcb7cfe97cdd6fbff8 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b533a7d8d97943609c6e28e3e36b78a4 in term 1.
I20240502 07:59:25.476375 22308 leader_election.cc:302] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, b533a7d8d97943609c6e28e3e36b78a4; no voters:
I20240502 07:59:25.476692 308 raft_consensus.cc:2781] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:25.476778 308 raft_consensus.cc:686] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Becoming Leader. State: Replica: b533a7d8d97943609c6e28e3e36b78a4, State: Running, Role: LEADER
I20240502 07:59:25.476903 308 consensus_queue.cc:237] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } }
I20240502 07:59:25.480221 22227 catalog_manager.cc:5484] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129). New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } }
I20240502 07:59:25.488860 2066 tablet_bootstrap.cc:492] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c: No bootstrap required, opened a new log
I20240502 07:59:25.488973 2066 ts_tablet_manager.cc:1367] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20240502 07:59:25.489512 2066 raft_consensus.cc:348] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:25.489655 2066 raft_consensus.cc:374] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:25.489691 2066 raft_consensus.cc:729] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Initialized, Role: FOLLOWER
I20240502 07:59:25.489804 2066 consensus_queue.cc:260] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:25.490114 2066 ts_tablet_manager.cc:1398] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
W20240502 07:59:25.512276 22569 consensus_peers.cc:462] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:25.518803 1257 raft_consensus.cc:482] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:25.518937 1257 raft_consensus.cc:504] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:25.519397 1257 leader_election.cc:288] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:25.519938 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0e03a628f9b4119936b884680202c0c" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231" is_pre_election: true
I20240502 07:59:25.520071 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0e03a628f9b4119936b884680202c0c" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:25.520164 24951 raft_consensus.cc:2454] T e0e03a628f9b4119936b884680202c0c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:25.520359 25083 raft_consensus.cc:2454] T e0e03a628f9b4119936b884680202c0c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 0.
I20240502 07:59:25.520869 22436 leader_election.cc:302] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:25.521070 1257 raft_consensus.cc:2781] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:25.521497 1257 raft_consensus.cc:482] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:25.521567 1257 raft_consensus.cc:3037] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:25.523129 1257 raft_consensus.cc:504] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:25.523552 1257 leader_election.cc:288] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Requested vote from peers 1aecef81733c47b3b468504934357231 (127.15.129.134:35995), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733)
I20240502 07:59:25.524024 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0e03a628f9b4119936b884680202c0c" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055"
I20240502 07:59:25.524164 25083 raft_consensus.cc:3037] T e0e03a628f9b4119936b884680202c0c P 227eec4b5f014d349661f6e0b39b9055 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:25.525223 25083 raft_consensus.cc:2454] T e0e03a628f9b4119936b884680202c0c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
I20240502 07:59:25.525694 22436 leader_election.cc:302] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, fabddb16c85944648a710ec52fc7244c; no voters:
I20240502 07:59:25.525961 1257 raft_consensus.cc:2781] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:25.526166 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0e03a628f9b4119936b884680202c0c" candidate_uuid: "fabddb16c85944648a710ec52fc7244c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aecef81733c47b3b468504934357231"
I20240502 07:59:25.526298 24951 raft_consensus.cc:3037] T e0e03a628f9b4119936b884680202c0c P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:25.526556 1257 raft_consensus.cc:686] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [term 1 LEADER]: Becoming Leader. State: Replica: fabddb16c85944648a710ec52fc7244c, State: Running, Role: LEADER
I20240502 07:59:25.526708 1257 consensus_queue.cc:237] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:25.527137 24951 raft_consensus.cc:2454] T e0e03a628f9b4119936b884680202c0c P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fabddb16c85944648a710ec52fc7244c in term 1.
W20240502 07:59:25.529577 22700 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111) [suppressed 53 similar messages]
I20240502 07:59:25.529632 22225 catalog_manager.cc:5484] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c reported cstate change: term changed from 0 to 1, leader changed from <none> to fabddb16c85944648a710ec52fc7244c (127.15.129.130). New cstate: current_term: 1 leader_uuid: "fabddb16c85944648a710ec52fc7244c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
W20240502 07:59:25.543865 25019 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111) [suppressed 112 similar messages]
W20240502 07:59:25.577376 22307 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111) [suppressed 92 similar messages]
W20240502 07:59:25.578099 22307 consensus_peers.cc:485] T e6cbb33529514aa0be60d0a2d70fe9a8 P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:25.590691 22569 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111) [suppressed 95 similar messages]
I20240502 07:59:25.596426 1673 raft_consensus.cc:482] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:25.596596 1673 raft_consensus.cc:504] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:25.597133 1673 leader_election.cc:288] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:25.597568 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317b0e49db674cb2bec93398c830bc85" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f" is_pre_election: true
I20240502 07:59:25.597712 22634 raft_consensus.cc:2454] T 317b0e49db674cb2bec93398c830bc85 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:25.597718 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317b0e49db674cb2bec93398c830bc85" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4" is_pre_election: true
I20240502 07:59:25.597895 22370 raft_consensus.cc:2454] T 317b0e49db674cb2bec93398c830bc85 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 0.
I20240502 07:59:25.597975 24888 leader_election.cc:302] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:25.598162 1673 raft_consensus.cc:2781] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:25.598207 1673 raft_consensus.cc:482] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:25.598245 1673 raft_consensus.cc:3037] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:25.599402 1673 raft_consensus.cc:504] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
W20240502 07:59:25.599586 25150 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111) [suppressed 107 similar messages]
I20240502 07:59:25.599913 1673 leader_election.cc:288] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Requested vote from peers e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:25.600360 22634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317b0e49db674cb2bec93398c830bc85" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e51b10af49484aaa8f0267aebc1bf27f"
I20240502 07:59:25.600476 22634 raft_consensus.cc:3037] T 317b0e49db674cb2bec93398c830bc85 P e51b10af49484aaa8f0267aebc1bf27f [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:25.600699 22370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "317b0e49db674cb2bec93398c830bc85" candidate_uuid: "1aecef81733c47b3b468504934357231" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b533a7d8d97943609c6e28e3e36b78a4"
W20240502 07:59:25.600773 25150 consensus_peers.cc:462] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:25.600803 22370 raft_consensus.cc:3037] T 317b0e49db674cb2bec93398c830bc85 P b533a7d8d97943609c6e28e3e36b78a4 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:25.601449 22634 raft_consensus.cc:2454] T 317b0e49db674cb2bec93398c830bc85 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:25.601742 24888 leader_election.cc:302] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aecef81733c47b3b468504934357231, e51b10af49484aaa8f0267aebc1bf27f; no voters:
I20240502 07:59:25.601904 22370 raft_consensus.cc:2454] T 317b0e49db674cb2bec93398c830bc85 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aecef81733c47b3b468504934357231 in term 1.
I20240502 07:59:25.602070 1673 raft_consensus.cc:2781] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:25.602165 1673 raft_consensus.cc:686] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Becoming Leader. State: Replica: 1aecef81733c47b3b468504934357231, State: Running, Role: LEADER
I20240502 07:59:25.602291 1673 consensus_queue.cc:237] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:25.606083 22223 catalog_manager.cc:5484] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aecef81733c47b3b468504934357231 (127.15.129.134). New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } }
W20240502 07:59:25.622695 22307 consensus_peers.cc:485] T 423d6be23b044130b0c916f995dffced P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.630812 25063 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:25.633194 25193 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:25.638420 22352 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:25.640106 24930 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
W20240502 07:59:25.643900 24887 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111) [suppressed 74 similar messages]
I20240502 07:59:25.649639 22614 tablet_service.cc:1429] Tablet server has 28 leaders and 0 scanners
W20240502 07:59:25.653668 22438 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111) [suppressed 82 similar messages]
W20240502 07:59:25.655287 22438 consensus_peers.cc:485] T eebad8d185b74842b576bddc1a7c7b0d P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.661983 22482 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
W20240502 07:59:25.667842 22307 consensus_peers.cc:485] T c2422c666baa48078030c43d1c0938f9 P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:25.674988 25150 consensus_peers.cc:485] T 8aa4489eaac144c7926c6be2f199c46d P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.690183 22745 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:25.692140 492 raft_consensus.cc:482] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b4e7eeb7f3f42119c0363a79febc54b)
I20240502 07:59:25.692250 492 raft_consensus.cc:504] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4b4e7eeb7f3f42119c0363a79febc54b" member_type: VOTER last_known_addr { host: "127.15.129.133" port: 41661 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } }
I20240502 07:59:25.692723 492 leader_election.cc:288] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661), 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733), b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535)
I20240502 07:59:25.692996 25084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05911d56b0344eb8a53aba5f4bf0170c" candidate_uuid: "43930dee0cd5498fa8e357a34dd71432" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "227eec4b5f014d349661f6e0b39b9055" is_pre_election: true
I20240502 07:59:25.693141 25084 raft_consensus.cc:2454] T 05911d56b0344eb8a53aba5f4bf0170c P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43930dee0cd5498fa8e357a34dd71432 in term 1.
W20240502 07:59:25.694365 25150 leader_election.cc:334] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:25.694730 25151 leader_election.cc:341] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): Not found: Tablet not found: 05911d56b0344eb8a53aba5f4bf0170c
I20240502 07:59:25.694811 25151 leader_election.cc:302] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 227eec4b5f014d349661f6e0b39b9055, 43930dee0cd5498fa8e357a34dd71432; no voters: 4b4e7eeb7f3f42119c0363a79febc54b, b533a7d8d97943609c6e28e3e36b78a4
I20240502 07:59:25.695008 492 raft_consensus.cc:2726] T 05911d56b0344eb8a53aba5f4bf0170c P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20240502 07:59:25.695916 24887 consensus_peers.cc:485] T d99bc35db172424bbc8536a1c95601fe P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:25.708875 25150 consensus_peers.cc:485] T 4017b918d5f1485aaaf05c764f08ad67 P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.739046 22370 consensus_queue.cc:237] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:25.740409 25215 raft_consensus.cc:1261] T 45925e9713cb4b748efde06f77cf3702 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:25.740407 24952 raft_consensus.cc:1261] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Refusing update from remote peer b533a7d8d97943609c6e28e3e36b78a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:25.740880 308 consensus_queue.cc:1035] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:25.741221 308 consensus_queue.cc:1035] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:25.743502 32390 raft_consensus.cc:2932] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:25.743610 24952 raft_consensus.cc:2932] T 45925e9713cb4b748efde06f77cf3702 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:25.746174 25215 raft_consensus.cc:2932] T 45925e9713cb4b748efde06f77cf3702 P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:25.747288 22223 catalog_manager.cc:5484] T 45925e9713cb4b748efde06f77cf3702 P b533a7d8d97943609c6e28e3e36b78a4 reported cstate change: config changed from index 6 to 7, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New cstate: current_term: 1 leader_uuid: "b533a7d8d97943609c6e28e3e36b78a4" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:25.753772 22614 tablet_service.cc:1514] Processing DeleteTablet for tablet 45925e9713cb4b748efde06f77cf3702 with delete_type TABLET_DATA_TOMBSTONED (TS e51b10af49484aaa8f0267aebc1bf27f not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:25.754041 2019 tablet_replica.cc:330] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:25.754184 2019 raft_consensus.cc:2229] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:25.754345 2019 raft_consensus.cc:2258] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:25.754977 2019 ts_tablet_manager.cc:1875] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:25.757164 2019 ts_tablet_manager.cc:1888] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:25.757241 2019 log.cc:1196] T 45925e9713cb4b748efde06f77cf3702 P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/45925e9713cb4b748efde06f77cf3702
I20240502 07:59:25.762467 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 45925e9713cb4b748efde06f77cf3702 (table rebalance_test_table_7 [id=255d8aedf8f845a1b8080105bc5b3a93]) successfully deleted
W20240502 07:59:25.778373 22700 consensus_peers.cc:485] T 6aee8f101c5e46df994dd15b49164a2b P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:25.789098 22438 consensus_peers.cc:485] T c08b5f0114744488a3065bfc4d04d993 P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240502 07:59:25.792846 25019 consensus_peers.cc:485] T 9f44c905bfaa4db093a1b39b6c21e03c P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.796866 1874 consensus_queue.cc:1035] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:25.799798 1874 consensus_queue.cc:1035] T 9be2b30a7d80488db57445063ee35020 P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 07:59:25.855841 25019 consensus_peers.cc:485] T a511340ea9e649c690b11afb9a35835f P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:25.858186 24887 consensus_peers.cc:485] T b5cd6d896bd74c59848db52a235b78b2 P 1aecef81733c47b3b468504934357231 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:25.875073 22700 consensus_peers.cc:462] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:25.875753 22569 consensus_peers.cc:485] T 648a25ebebe44061bca2844ec5b048c5 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:25.891322 24930 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 07:59:25.901444 22614 tablet_service.cc:1429] Tablet server has 28 leaders and 0 scanners
I20240502 07:59:25.903643 25193 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:25.904477 22482 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:25.919847 22352 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:25.920348 22745 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:25.934064 25063 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
W20240502 07:59:25.945945 22569 consensus_peers.cc:462] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:25.949199 22438 consensus_peers.cc:485] T e8534d84d3ce44b6aa74767f2da695cf P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:25.978204 25150 consensus_peers.cc:462] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:26.008030 1245 consensus_queue.cc:1035] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:26.010182 2103 consensus_queue.cc:1035] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:26.013376 1257 consensus_queue.cc:1035] T e0e03a628f9b4119936b884680202c0c P fabddb16c85944648a710ec52fc7244c [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:26.015663 2103 consensus_queue.cc:1035] T 4d06f144440146fcb7cfe97cdd6fbff8 P b533a7d8d97943609c6e28e3e36b78a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:26.020916 1673 consensus_queue.cc:1035] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:26.025532 601 consensus_queue.cc:1035] T 317b0e49db674cb2bec93398c830bc85 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b533a7d8d97943609c6e28e3e36b78a4" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:26.124632 25063 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:26.125299 25193 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:26.135826 22482 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
W20240502 07:59:26.136603 25150 consensus_peers.cc:485] T efdd040f4d6748879b399081416b0c42 P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:26.143420 22352 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:26.144006 24930 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 07:59:26.167462 22614 tablet_service.cc:1429] Tablet server has 28 leaders and 0 scanners
I20240502 07:59:26.172292 22745 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
W20240502 07:59:26.197587 22216 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111) [suppressed 64 similar messages]
I20240502 07:59:26.220216 22764 consensus_queue.cc:237] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } }
I20240502 07:59:26.221441 25084 raft_consensus.cc:1261] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:26.221509 25215 raft_consensus.cc:1261] T 895f9a31785d4558acd3b64904a645ca P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Refusing update from remote peer f00f3a1876444ad2bf9c919068ab8014: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20240502 07:59:26.221896 1874 consensus_queue.cc:1035] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:26.222074 31905 consensus_queue.cc:1035] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:26.223767 31905 raft_consensus.cc:2932] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:26.224205 25084 raft_consensus.cc:2932] T 895f9a31785d4558acd3b64904a645ca P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:26.225531 25215 raft_consensus.cc:2932] T 895f9a31785d4558acd3b64904a645ca P 43930dee0cd5498fa8e357a34dd71432 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:26.227157 22221 catalog_manager.cc:5484] T 895f9a31785d4558acd3b64904a645ca P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index 6 to 7, VOTER e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131) evicted. New cstate: current_term: 1 leader_uuid: "f00f3a1876444ad2bf9c919068ab8014" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "227eec4b5f014d349661f6e0b39b9055" member_type: VOTER last_known_addr { host: "127.15.129.135" port: 42733 } } peers { permanent_uuid: "43930dee0cd5498fa8e357a34dd71432" member_type: VOTER last_known_addr { host: "127.15.129.136" port: 36591 } } }
I20240502 07:59:26.232513 22614 tablet_service.cc:1514] Processing DeleteTablet for tablet 895f9a31785d4558acd3b64904a645ca with delete_type TABLET_DATA_TOMBSTONED (TS e51b10af49484aaa8f0267aebc1bf27f not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:47186
I20240502 07:59:26.232802 2019 tablet_replica.cc:330] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f: stopping tablet replica
I20240502 07:59:26.232992 2019 raft_consensus.cc:2229] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:26.233170 2019 raft_consensus.cc:2258] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:26.233762 2019 ts_tablet_manager.cc:1875] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:26.236002 2019 ts_tablet_manager.cc:1888] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.6
I20240502 07:59:26.236088 2019 log.cc:1196] T 895f9a31785d4558acd3b64904a645ca P e51b10af49484aaa8f0267aebc1bf27f: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/wals/895f9a31785d4558acd3b64904a645ca
I20240502 07:59:26.236626 22217 catalog_manager.cc:4830] TS e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): tablet 895f9a31785d4558acd3b64904a645ca (table rebalance_test_table_15 [id=4e81cbd084694ac8b1db3f5574dde601]) successfully deleted
W20240502 07:59:26.269237 22700 consensus_peers.cc:485] T 5ee489f17d80406d8fd5c2909612f456 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:26.365811 24930 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 07:59:26.372771 25063 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:26.373076 22482 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:26.378865 22352 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:26.378957 22614 tablet_service.cc:1429] Tablet server has 28 leaders and 0 scanners
I20240502 07:59:26.383816 22745 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:26.399644 25193 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
W20240502 07:59:26.424103 22700 consensus_peers.cc:462] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:26.466957 22634 consensus_queue.cc:237] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } }
I20240502 07:59:26.468544 22763 raft_consensus.cc:1261] T fe087f9b25c84749b7cfba34800da679 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:26.468633 22499 raft_consensus.cc:1261] T fe087f9b25c84749b7cfba34800da679 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer e51b10af49484aaa8f0267aebc1bf27f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:26.470187 2312 consensus_queue.cc:1035] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20240502 07:59:26.472841 25150 consensus_peers.cc:462] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:26.473277 1933 consensus_queue.cc:1035] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:26.474066 22763 raft_consensus.cc:2932] T fe087f9b25c84749b7cfba34800da679 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:26.474704 1024 raft_consensus.cc:2932] T fe087f9b25c84749b7cfba34800da679 P e51b10af49484aaa8f0267aebc1bf27f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:26.478135 22221 catalog_manager.cc:5484] T fe087f9b25c84749b7cfba34800da679 P f00f3a1876444ad2bf9c919068ab8014 reported cstate change: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New cstate: current_term: 1 leader_uuid: "e51b10af49484aaa8f0267aebc1bf27f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:26.483440 24930 tablet_service.cc:1514] Processing DeleteTablet for tablet fe087f9b25c84749b7cfba34800da679 with delete_type TABLET_DATA_TOMBSTONED (TS 1aecef81733c47b3b468504934357231 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:37828
I20240502 07:59:26.485160 22499 raft_consensus.cc:2932] T fe087f9b25c84749b7cfba34800da679 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 1aecef81733c47b3b468504934357231 (127.15.129.134) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "e51b10af49484aaa8f0267aebc1bf27f" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 43703 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } }
I20240502 07:59:26.486743 2315 tablet_replica.cc:330] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231: stopping tablet replica
I20240502 07:59:26.486898 2315 raft_consensus.cc:2229] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:26.487044 2315 raft_consensus.cc:2258] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:26.489482 2315 ts_tablet_manager.cc:1875] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20240502 07:59:26.491391 22569 consensus_peers.cc:462] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:26.492425 2315 ts_tablet_manager.cc:1888] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:26.492537 2315 log.cc:1196] T fe087f9b25c84749b7cfba34800da679 P 1aecef81733c47b3b468504934357231: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/wals/fe087f9b25c84749b7cfba34800da679
I20240502 07:59:26.493042 22217 catalog_manager.cc:4830] TS 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): tablet fe087f9b25c84749b7cfba34800da679 (table rebalancer_extra_table_46 [id=ae9e0c6dd5794a02bcfb8899f4d26273]) successfully deleted
I20240502 07:59:26.636531 24930 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 07:59:26.646443 25063 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:26.648001 22482 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
I20240502 07:59:26.652472 25193 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
I20240502 07:59:26.666028 22614 tablet_service.cc:1429] Tablet server has 28 leaders and 0 scanners
I20240502 07:59:26.673413 22745 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:26.684698 22352 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
W20240502 07:59:26.744441 22700 consensus_peers.cc:485] T 521231ade647437683f225bedbd8d06b P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240502 07:59:26.754158 24951 consensus_queue.cc:237] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } }
I20240502 07:59:26.755429 22501 raft_consensus.cc:1261] T 461098a3f8e64fe89a8603f78dc36dd2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:26.755429 22763 raft_consensus.cc:1261] T 461098a3f8e64fe89a8603f78dc36dd2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Refusing update from remote peer 1aecef81733c47b3b468504934357231: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20240502 07:59:26.755827 1673 consensus_queue.cc:1035] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:26.756031 2357 consensus_queue.cc:1035] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20240502 07:59:26.757789 1673 raft_consensus.cc:2932] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:26.758232 22501 raft_consensus.cc:2932] T 461098a3f8e64fe89a8603f78dc36dd2 P fabddb16c85944648a710ec52fc7244c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:26.760113 22763 raft_consensus.cc:2932] T 461098a3f8e64fe89a8603f78dc36dd2 P f00f3a1876444ad2bf9c919068ab8014 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:26.762615 22225 catalog_manager.cc:5484] T 461098a3f8e64fe89a8603f78dc36dd2 P 1aecef81733c47b3b468504934357231 reported cstate change: config changed from index 2 to 3, VOTER 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135) evicted. New cstate: current_term: 1 leader_uuid: "1aecef81733c47b3b468504934357231" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "fabddb16c85944648a710ec52fc7244c" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 39625 } } peers { permanent_uuid: "1aecef81733c47b3b468504934357231" member_type: VOTER last_known_addr { host: "127.15.129.134" port: 35995 } } peers { permanent_uuid: "f00f3a1876444ad2bf9c919068ab8014" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 41673 } } }
I20240502 07:59:26.766525 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet 461098a3f8e64fe89a8603f78dc36dd2 with delete_type TABLET_DATA_TOMBSTONED (TS 227eec4b5f014d349661f6e0b39b9055 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:42038
I20240502 07:59:26.767246 2392 tablet_replica.cc:330] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055: stopping tablet replica
I20240502 07:59:26.767452 2392 raft_consensus.cc:2229] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240502 07:59:26.767633 2392 raft_consensus.cc:2258] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240502 07:59:26.768097 2392 ts_tablet_manager.cc:1875] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240502 07:59:26.771030 2392 ts_tablet_manager.cc:1888] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20240502 07:59:26.771107 2392 log.cc:1196] T 461098a3f8e64fe89a8603f78dc36dd2 P 227eec4b5f014d349661f6e0b39b9055: Deleting WAL directory at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/wals/461098a3f8e64fe89a8603f78dc36dd2
I20240502 07:59:26.771682 22214 catalog_manager.cc:4830] TS 227eec4b5f014d349661f6e0b39b9055 (127.15.129.135:42733): tablet 461098a3f8e64fe89a8603f78dc36dd2 (table rebalancer_extra_table_18 [id=7472e589acd942d3a925f8309b076ca6]) successfully deleted
W20240502 07:59:26.776456 25150 consensus_peers.cc:485] T 712a88cd545745e8937d295266e85f04 P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:26.889577 25019 consensus_peers.cc:485] T 4cd6c14c7c8b4ad29f2107e94fd11acc P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:26.898267 22307 consensus_peers.cc:485] T 52e8c7b14d4b45fa9f7b28e755e1f5df P b533a7d8d97943609c6e28e3e36b78a4 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:26.898586 22569 consensus_peers.cc:462] T 72964b1c935441ca840a46ff79921cc3 P e51b10af49484aaa8f0267aebc1bf27f -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:26.952569 25063 tablet_service.cc:1429] Tablet server has 18 leaders and 0 scanners
I20240502 07:59:26.953141 24930 tablet_service.cc:1429] Tablet server has 21 leaders and 0 scanners
I20240502 07:59:26.959232 25193 tablet_service.cc:1429] Tablet server has 20 leaders and 0 scanners
W20240502 07:59:26.963086 22700 consensus_peers.cc:462] T c5a30e1ebd7747eb911fba8437e42077 P f00f3a1876444ad2bf9c919068ab8014 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:26.973346 22614 tablet_service.cc:1429] Tablet server has 28 leaders and 0 scanners
I20240502 07:59:26.973632 22352 tablet_service.cc:1429] Tablet server has 17 leaders and 0 scanners
I20240502 07:59:26.977511 22745 tablet_service.cc:1429] Tablet server has 15 leaders and 0 scanners
I20240502 07:59:26.983211 22482 tablet_service.cc:1429] Tablet server has 19 leaders and 0 scanners
W20240502 07:59:27.000627 25019 consensus_peers.cc:485] T 751f8c9cbf684b01b90e7bd26685f8bf P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:27.001195 22216 catalog_manager.cc:4573] TS 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): CreateTablet RPC failed for tablet c08b5f0114744488a3065bfc4d04d993: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:27.017585 25150 consensus_peers.cc:462] T 89889b6805684edeb79afd3fa3a55dea P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Unable to start Tablet Copy on peer: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
W20240502 07:59:27.019565 25019 consensus_peers.cc:485] T 8fcb3fef170343d494d8dfbba33052d9 P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:27.036062 22438 consensus_peers.cc:485] T 79a67cfa3c554f2d85a520bb8f5ccc81 P fabddb16c85944648a710ec52fc7244c -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:27.062039 25019 consensus_peers.cc:485] T 9f916726456f4aa8936a9366468bc471 P 227eec4b5f014d349661f6e0b39b9055 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tools/rebalancer_tool-test.cc:1203: Failure
Value of: s.ok()
Actual: false
Expected: true
Runtime error: /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu: process exited with non-zero status 1
stdout: Per-server replica distribution summary:
Statistic | Value
-----------------------+-----------
Minimum Replica Count | 0
Maximum Replica Count | 45
Average Replica Count | 22.875000
Per-table replica distribution summary:
Replica Skew | Value
--------------+----------
Minimum | 1
Maximum | 3
Average | 2.904762
stderr: W20240502 07:58:55.735183 25269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:58:55.735494 25269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:58:55.739769 25269 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20240502 07:58:56.243736 25269 rebalancer_tool.cc:215] running whole-cluster rebalancing
W20240502 07:58:56.598726 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table ad840b2a5e7149b78acd24dc98f05685: could not find any suitable replica to move from server 4b4e7eeb7f3f42119c0363a79febc54b to server e51b10af49484aaa8f0267aebc1bf27f
W20240502 07:58:56.599520 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table ad840b2a5e7149b78acd24dc98f05685: could not find any suitable replica to move from server 4b4e7eeb7f3f42119c0363a79febc54b to server fabddb16c85944648a710ec52fc7244c
W20240502 07:58:56.599678 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table bef7e70a9994447ca2f4a9a43821470d: could not find any suitable replica to move from server fabddb16c85944648a710ec52fc7244c to server 1aecef81733c47b3b468504934357231
W20240502 07:58:56.599879 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 0937fa4568b0489ebd23bdd0b698a759: could not find any suitable replica to move from server fabddb16c85944648a710ec52fc7244c to server 4b4e7eeb7f3f42119c0363a79febc54b
W20240502 07:58:56.599934 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 6fc58da1a2c74e7a90d58e172a67ce50: could not find any suitable replica to move from server f00f3a1876444ad2bf9c919068ab8014 to server 227eec4b5f014d349661f6e0b39b9055
W20240502 07:58:56.599972 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 4e81cbd084694ac8b1db3f5574dde601: could not find any suitable replica to move from server e51b10af49484aaa8f0267aebc1bf27f to server 43930dee0cd5498fa8e357a34dd71432
W20240502 07:58:56.600019 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 255d8aedf8f845a1b8080105bc5b3a93: could not find any suitable replica to move from server e51b10af49484aaa8f0267aebc1bf27f to server 43930dee0cd5498fa8e357a34dd71432
W20240502 07:58:56.600057 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 2c887f37ddf349f3bbec2b1330f05fc7: could not find any suitable replica to move from server b533a7d8d97943609c6e28e3e36b78a4 to server 1aecef81733c47b3b468504934357231
W20240502 07:58:56.600116 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table c952b8d018e640108111542771b4f142: could not find any suitable replica to move from server f00f3a1876444ad2bf9c919068ab8014 to server 227eec4b5f014d349661f6e0b39b9055
W20240502 07:58:56.600237 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table be3f9b3a6e5147b0afe27972b50219ce: could not find any suitable replica to move from server e51b10af49484aaa8f0267aebc1bf27f to server 227eec4b5f014d349661f6e0b39b9055
W20240502 07:58:56.600287 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 9b69b771dcfb4a00a937ed9203245903: could not find any suitable replica to move from server fabddb16c85944648a710ec52fc7244c to server 1aecef81733c47b3b468504934357231
W20240502 07:58:56.600333 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table d581572e1d9e4b579052fa1e17f5a037: could not find any suitable replica to move from server f00f3a1876444ad2bf9c919068ab8014 to server 4b4e7eeb7f3f42119c0363a79febc54b
W20240502 07:58:56.600376 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 80b9c0f932754699a4f41d2c2188b70b: could not find any suitable replica to move from server fabddb16c85944648a710ec52fc7244c to server 43930dee0cd5498fa8e357a34dd71432
W20240502 07:58:56.600420 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 1ec511fc66814cefa73a9b7acff4eea9: could not find any suitable replica to move from server e51b10af49484aaa8f0267aebc1bf27f to server 43930dee0cd5498fa8e357a34dd71432
W20240502 07:58:56.600821 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 4ae19d06ad284921b8f132a6449508a4: could not find any suitable replica to move from server f00f3a1876444ad2bf9c919068ab8014 to server 1aecef81733c47b3b468504934357231
W20240502 07:58:56.600863 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 490b2fcdef13484180791ae6a0d1907d: could not find any suitable replica to move from server e51b10af49484aaa8f0267aebc1bf27f to server 227eec4b5f014d349661f6e0b39b9055
W20240502 07:58:56.600965 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 74c99be4bec647c09e6275ef44e45334: could not find any suitable replica to move from server e51b10af49484aaa8f0267aebc1bf27f to server 4b4e7eeb7f3f42119c0363a79febc54b
W20240502 07:58:56.601024 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 4e87626760fa4d8098e55dfa6d1c55a9: could not find any suitable replica to move from server f00f3a1876444ad2bf9c919068ab8014 to server 227eec4b5f014d349661f6e0b39b9055
W20240502 07:58:56.601058 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 423ebbc6961345159cc3d4c9668fe1dd: could not find any suitable replica to move from server e51b10af49484aaa8f0267aebc1bf27f to server 43930dee0cd5498fa8e357a34dd71432
W20240502 07:58:56.601089 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 144806269704437bbac295cdc492d042: could not find any suitable replica to move from server e51b10af49484aaa8f0267aebc1bf27f to server 1aecef81733c47b3b468504934357231
W20240502 07:58:56.601126 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table c09f49e6f05e4cab88a126bdd611a5f2: could not find any suitable replica to move from server b533a7d8d97943609c6e28e3e36b78a4 to server 43930dee0cd5498fa8e357a34dd71432
W20240502 07:58:56.601159 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table d581572e1d9e4b579052fa1e17f5a037: could not find any suitable replica to move from server fabddb16c85944648a710ec52fc7244c to server 1aecef81733c47b3b468504934357231
W20240502 07:58:56.601191 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table bef7e70a9994447ca2f4a9a43821470d: could not find any suitable replica to move from server f00f3a1876444ad2bf9c919068ab8014 to server 227eec4b5f014d349661f6e0b39b9055
W20240502 07:58:56.601223 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 4ae19d06ad284921b8f132a6449508a4: could not find any suitable replica to move from server fabddb16c85944648a710ec52fc7244c to server 4b4e7eeb7f3f42119c0363a79febc54b
W20240502 07:58:56.601254 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table be3f9b3a6e5147b0afe27972b50219ce: could not find any suitable replica to move from server f00f3a1876444ad2bf9c919068ab8014 to server 1aecef81733c47b3b468504934357231
W20240502 07:58:56.601285 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 9b69b771dcfb4a00a937ed9203245903: could not find any suitable replica to move from server e51b10af49484aaa8f0267aebc1bf27f to server 43930dee0cd5498fa8e357a34dd71432
W20240502 07:58:56.601372 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 490b2fcdef13484180791ae6a0d1907d: could not find any suitable replica to move from server fabddb16c85944648a710ec52fc7244c to server 4b4e7eeb7f3f42119c0363a79febc54b
W20240502 07:58:56.601418 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 255d8aedf8f845a1b8080105bc5b3a93: could not find any suitable replica to move from server b533a7d8d97943609c6e28e3e36b78a4 to server 227eec4b5f014d349661f6e0b39b9055
I20240502 07:58:56.651254 25269 rebalancer_tool.cc:1074] tablet e6a499fd83554a2c8e3e837aedddb61b: 'f00f3a1876444ad2bf9c919068ab8014' -> '227eec4b5f014d349661f6e0b39b9055' move scheduled
I20240502 07:58:56.681788 25269 rebalancer_tool.cc:1074] tablet c5a30e1ebd7747eb911fba8437e42077: 'e51b10af49484aaa8f0267aebc1bf27f' -> '1aecef81733c47b3b468504934357231' move scheduled
I20240502 07:58:56.715380 25269 rebalancer_tool.cc:1074] tablet ce0f9b48d2db4855a86eb20955fdc853: 'fabddb16c85944648a710ec52fc7244c' -> '43930dee0cd5498fa8e357a34dd71432' move scheduled
I20240502 07:58:56.753091 25269 rebalancer_tool.cc:1074] tablet 257f0f4dca624c859d3b06cfd4246536: 'b533a7d8d97943609c6e28e3e36b78a4' -> '4b4e7eeb7f3f42119c0363a79febc54b' move scheduled
I20240502 07:58:56.786461 25269 rebalancer_tool.cc:1074] tablet d4c1fadf79ab475b83e3303918f96d46: 'f00f3a1876444ad2bf9c919068ab8014' -> '227eec4b5f014d349661f6e0b39b9055' move scheduled
I20240502 07:58:56.815033 25269 rebalancer_tool.cc:1074] tablet 02449bcab2294956918d1889485cfd5f: 'e51b10af49484aaa8f0267aebc1bf27f' -> '1aecef81733c47b3b468504934357231' move scheduled
I20240502 07:58:56.841069 25269 rebalancer_tool.cc:1074] tablet 72964b1c935441ca840a46ff79921cc3: 'fabddb16c85944648a710ec52fc7244c' -> '43930dee0cd5498fa8e357a34dd71432' move scheduled
I20240502 07:58:56.882261 25269 rebalancer_tool.cc:1074] tablet d6dfeeb6b16a4f92bc756bf8933c7fb1: 'b533a7d8d97943609c6e28e3e36b78a4' -> '4b4e7eeb7f3f42119c0363a79febc54b' move scheduled
I20240502 07:58:56.918833 25269 rebalancer_tool.cc:1074] tablet d9c0c75053be41f59786f1cf4fb3062c: 'f00f3a1876444ad2bf9c919068ab8014' -> '227eec4b5f014d349661f6e0b39b9055' move scheduled
I20240502 07:58:56.944502 25269 rebalancer_tool.cc:1074] tablet 4e202e9cf0154eb9814a7e3ad93f1e42: 'e51b10af49484aaa8f0267aebc1bf27f' -> '1aecef81733c47b3b468504934357231' move scheduled
I20240502 07:58:56.982090 25269 rebalancer_tool.cc:1074] tablet 648a25ebebe44061bca2844ec5b048c5: 'fabddb16c85944648a710ec52fc7244c' -> '4b4e7eeb7f3f42119c0363a79febc54b' move scheduled
I20240502 07:58:57.006953 25269 rebalancer_tool.cc:1074] tablet c9b0511ea0124d84ac5d3996bd1efcd2: 'b533a7d8d97943609c6e28e3e36b78a4' -> '43930dee0cd5498fa8e357a34dd71432' move scheduled
I20240502 07:58:57.063680 25269 rebalancer_tool.cc:1074] tablet fa14f8f327cb4221aabeedac16d258b6: 'f00f3a1876444ad2bf9c919068ab8014' -> '227eec4b5f014d349661f6e0b39b9055' move scheduled
I20240502 07:58:57.130363 25269 rebalancer_tool.cc:1074] tablet c2422c666baa48078030c43d1c0938f9: 'e51b10af49484aaa8f0267aebc1bf27f' -> '4b4e7eeb7f3f42119c0363a79febc54b' move scheduled
I20240502 07:58:57.177122 25269 rebalancer_tool.cc:1074] tablet 45925e9713cb4b748efde06f77cf3702: 'fabddb16c85944648a710ec52fc7244c' -> '1aecef81733c47b3b468504934357231' move scheduled
I20240502 07:58:57.227734 25269 rebalancer_tool.cc:1074] tablet b9557af8565d4588b119f1915e76c3dc: 'b533a7d8d97943609c6e28e3e36b78a4' -> '43930dee0cd5498fa8e357a34dd71432' move scheduled
I20240502 07:58:57.311381 25269 rebalancer_tool.cc:1074] tablet 4dc697950c8140878b50550fe0ff458d: 'f00f3a1876444ad2bf9c919068ab8014' -> '4b4e7eeb7f3f42119c0363a79febc54b' move scheduled
I20240502 07:58:57.351054 25269 rebalancer_tool.cc:1074] tablet 92162f8f5fcb4baf96fa8004b7b41b9b: 'e51b10af49484aaa8f0267aebc1bf27f' -> '227eec4b5f014d349661f6e0b39b9055' move scheduled
I20240502 07:58:57.381647 25269 rebalancer_tool.cc:1074] tablet 41b4d8a50f684c19988a4cf1ee40921c: 'fabddb16c85944648a710ec52fc7244c' -> '1aecef81733c47b3b468504934357231' move scheduled
I20240502 07:58:57.415692 25269 rebalancer_tool.cc:1074] tablet 6d956ab18c8e479a9211afa3563dbe6f: 'b533a7d8d97943609c6e28e3e36b78a4' -> '43930dee0cd5498fa8e357a34dd71432' move scheduled
I20240502 07:58:59.772464 25269 rebalancer_tool.cc:1123] tablet b9557af8565d4588b119f1915e76c3dc: 'b533a7d8d97943609c6e28e3e36b78a4' -> '43930dee0cd5498fa8e357a34dd71432' move is abandoned: Not found: b9557af8565d4588b119f1915e76c3dc: tablet not found
I20240502 07:59:00.110419 25269 rebalancer_tool.cc:1123] tablet e6a499fd83554a2c8e3e837aedddb61b: 'f00f3a1876444ad2bf9c919068ab8014' -> '227eec4b5f014d349661f6e0b39b9055' move is abandoned: Not found: e6a499fd83554a2c8e3e837aedddb61b: tablet not found
I20240502 07:59:00.567075 25269 rebalancer_tool.cc:1123] tablet d6dfeeb6b16a4f92bc756bf8933c7fb1: 'b533a7d8d97943609c6e28e3e36b78a4' -> '4b4e7eeb7f3f42119c0363a79febc54b' move is abandoned: Not found: d6dfeeb6b16a4f92bc756bf8933c7fb1: tablet not found
I20240502 07:59:00.846365 25269 rebalancer_tool.cc:1123] tablet d4c1fadf79ab475b83e3303918f96d46: 'f00f3a1876444ad2bf9c919068ab8014' -> '227eec4b5f014d349661f6e0b39b9055' move is abandoned: Not found: d4c1fadf79ab475b83e3303918f96d46: tablet not found
I20240502 07:59:01.248100 25269 rebalancer_tool.cc:1123] tablet c9b0511ea0124d84ac5d3996bd1efcd2: 'b533a7d8d97943609c6e28e3e36b78a4' -> '43930dee0cd5498fa8e357a34dd71432' move is abandoned: Not found: c9b0511ea0124d84ac5d3996bd1efcd2: tablet not found
I20240502 07:59:01.248905 25269 rebalancer_tool.cc:1123] tablet fa14f8f327cb4221aabeedac16d258b6: 'f00f3a1876444ad2bf9c919068ab8014' -> '227eec4b5f014d349661f6e0b39b9055' move is abandoned: Not found: fa14f8f327cb4221aabeedac16d258b6: tablet not found
I20240502 07:59:01.249001 25269 rebalancer_tool.cc:756] re-synchronizing cluster state
W20240502 07:59:01.710654 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 1ec511fc66814cefa73a9b7acff4eea9: could not find any suitable replica to move from server e51b10af49484aaa8f0267aebc1bf27f to server 43930dee0cd5498fa8e357a34dd71432
W20240502 07:59:01.710762 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 80b9c0f932754699a4f41d2c2188b70b: could not find any suitable replica to move from server f00f3a1876444ad2bf9c919068ab8014 to server 227eec4b5f014d349661f6e0b39b9055
W20240502 07:59:01.710888 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 97589e13ee074648b68057aa77a8d10d: could not find any suitable replica to move from server 43930dee0cd5498fa8e357a34dd71432 to server e51b10af49484aaa8f0267aebc1bf27f
W20240502 07:59:01.710935 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 2c887f37ddf349f3bbec2b1330f05fc7: could not find any suitable replica to move from server fabddb16c85944648a710ec52fc7244c to server 1aecef81733c47b3b468504934357231
W20240502 07:59:01.711022 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table ad840b2a5e7149b78acd24dc98f05685: could not find any suitable replica to move from server 43930dee0cd5498fa8e357a34dd71432 to server e51b10af49484aaa8f0267aebc1bf27f
W20240502 07:59:01.711099 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 9b69b771dcfb4a00a937ed9203245903: could not find any suitable replica to move from server e51b10af49484aaa8f0267aebc1bf27f to server 43930dee0cd5498fa8e357a34dd71432
W20240502 07:59:01.711151 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 1ec511fc66814cefa73a9b7acff4eea9: could not find any suitable replica to move from server e51b10af49484aaa8f0267aebc1bf27f to server 1aecef81733c47b3b468504934357231
W20240502 07:59:01.711197 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 255d8aedf8f845a1b8080105bc5b3a93: could not find any suitable replica to move from server b533a7d8d97943609c6e28e3e36b78a4 to server 43930dee0cd5498fa8e357a34dd71432
W20240502 07:59:01.711253 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 4e87626760fa4d8098e55dfa6d1c55a9: could not find any suitable replica to move from server b533a7d8d97943609c6e28e3e36b78a4 to server 227eec4b5f014d349661f6e0b39b9055
W20240502 07:59:01.711315 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 490b2fcdef13484180791ae6a0d1907d: could not find any suitable replica to move from server fabddb16c85944648a710ec52fc7244c to server 4b4e7eeb7f3f42119c0363a79febc54b
W20240502 07:59:01.711377 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 4e81cbd084694ac8b1db3f5574dde601: could not find any suitable replica to move from server f00f3a1876444ad2bf9c919068ab8014 to server 4b4e7eeb7f3f42119c0363a79febc54b
W20240502 07:59:01.711450 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 423ebbc6961345159cc3d4c9668fe1dd: could not find any suitable replica to move from server f00f3a1876444ad2bf9c919068ab8014 to server 1aecef81733c47b3b468504934357231
W20240502 07:59:01.711503 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 72f8b22955164639b2802532262e4e2f: could not find any suitable replica to move from server b533a7d8d97943609c6e28e3e36b78a4 to server fabddb16c85944648a710ec52fc7244c
W20240502 07:59:01.711551 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table be3f9b3a6e5147b0afe27972b50219ce: could not find any suitable replica to move from server fabddb16c85944648a710ec52fc7244c to server 227eec4b5f014d349661f6e0b39b9055
W20240502 07:59:01.711660 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 4e81cbd084694ac8b1db3f5574dde601: could not find any suitable replica to move from server e51b10af49484aaa8f0267aebc1bf27f to server 43930dee0cd5498fa8e357a34dd71432
W20240502 07:59:01.711719 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 423ebbc6961345159cc3d4c9668fe1dd: could not find any suitable replica to move from server e51b10af49484aaa8f0267aebc1bf27f to server 227eec4b5f014d349661f6e0b39b9055
W20240502 07:59:01.711757 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table d17b69c6d9f54b1abe913a18b35cbbe5: could not find any suitable replica to move from server 4b4e7eeb7f3f42119c0363a79febc54b to server fabddb16c85944648a710ec52fc7244c
I20240502 07:59:01.742586 25269 rebalancer_tool.cc:1074] tablet cba71d2a28984e84b1950d59ec2e5a6a: '227eec4b5f014d349661f6e0b39b9055' -> 'f00f3a1876444ad2bf9c919068ab8014' move scheduled
I20240502 07:59:01.768115 25269 rebalancer_tool.cc:1074] tablet f49fcc586e6f498e93723d97b99c077e: '43930dee0cd5498fa8e357a34dd71432' -> 'b533a7d8d97943609c6e28e3e36b78a4' move scheduled
I20240502 07:59:01.799429 25269 rebalancer_tool.cc:1074] tablet 17421eec61584fbcac13f441922269bc: 'f00f3a1876444ad2bf9c919068ab8014' -> '227eec4b5f014d349661f6e0b39b9055' move scheduled
I20240502 07:59:01.834604 25269 rebalancer_tool.cc:1074] tablet 4840fa82fcb241f391c4890fb735fe9a: 'b533a7d8d97943609c6e28e3e36b78a4' -> '4b4e7eeb7f3f42119c0363a79febc54b' move scheduled
I20240502 07:59:01.869001 25269 rebalancer_tool.cc:1074] tablet 847f3a2601d44898a14a0b35409981ad: 'f00f3a1876444ad2bf9c919068ab8014' -> '43930dee0cd5498fa8e357a34dd71432' move scheduled
I20240502 07:59:01.896270 25269 rebalancer_tool.cc:1074] tablet 01d9a73be1fd472db39860e538b9c97e: '227eec4b5f014d349661f6e0b39b9055' -> 'b533a7d8d97943609c6e28e3e36b78a4' move scheduled
I20240502 07:59:03.018784 25269 rebalancer_tool.cc:1123] tablet 6d956ab18c8e479a9211afa3563dbe6f: 'b533a7d8d97943609c6e28e3e36b78a4' -> '43930dee0cd5498fa8e357a34dd71432' move is abandoned: Not found: 6d956ab18c8e479a9211afa3563dbe6f: tablet not found
I20240502 07:59:03.223632 25269 rebalancer_tool.cc:1123] tablet 4dc697950c8140878b50550fe0ff458d: 'f00f3a1876444ad2bf9c919068ab8014' -> '4b4e7eeb7f3f42119c0363a79febc54b' move is abandoned: Not found: 4dc697950c8140878b50550fe0ff458d: tablet not found
I20240502 07:59:03.237600 25269 rebalancer_tool.cc:1136] tablet 02449bcab2294956918d1889485cfd5f: 'e51b10af49484aaa8f0267aebc1bf27f' -> '1aecef81733c47b3b468504934357231' move completed: OK
I20240502 07:59:03.429513 25269 rebalancer_tool.cc:1123] tablet 257f0f4dca624c859d3b06cfd4246536: 'b533a7d8d97943609c6e28e3e36b78a4' -> '4b4e7eeb7f3f42119c0363a79febc54b' move is abandoned: Not found: 257f0f4dca624c859d3b06cfd4246536: tablet not found
I20240502 07:59:03.447381 25269 rebalancer_tool.cc:1136] tablet 72964b1c935441ca840a46ff79921cc3: 'fabddb16c85944648a710ec52fc7244c' -> '43930dee0cd5498fa8e357a34dd71432' move completed: OK
I20240502 07:59:03.454448 25269 rebalancer_tool.cc:1136] tablet ce0f9b48d2db4855a86eb20955fdc853: 'fabddb16c85944648a710ec52fc7244c' -> '43930dee0cd5498fa8e357a34dd71432' move completed: OK
I20240502 07:59:03.462471 25269 rebalancer_tool.cc:1136] tablet c2422c666baa48078030c43d1c0938f9: 'e51b10af49484aaa8f0267aebc1bf27f' -> '4b4e7eeb7f3f42119c0363a79febc54b' move completed: OK
I20240502 07:59:03.468933 25269 rebalancer_tool.cc:1136] tablet 92162f8f5fcb4baf96fa8004b7b41b9b: 'e51b10af49484aaa8f0267aebc1bf27f' -> '227eec4b5f014d349661f6e0b39b9055' move completed: OK
I20240502 07:59:03.475461 25269 rebalancer_tool.cc:1136] tablet 648a25ebebe44061bca2844ec5b048c5: 'fabddb16c85944648a710ec52fc7244c' -> '4b4e7eeb7f3f42119c0363a79febc54b' move completed: OK
I20240502 07:59:03.484719 25269 rebalancer_tool.cc:1136] tablet c5a30e1ebd7747eb911fba8437e42077: 'e51b10af49484aaa8f0267aebc1bf27f' -> '1aecef81733c47b3b468504934357231' move completed: OK
I20240502 07:59:03.490792 25269 rebalancer_tool.cc:1136] tablet d9c0c75053be41f59786f1cf4fb3062c: 'f00f3a1876444ad2bf9c919068ab8014' -> '227eec4b5f014d349661f6e0b39b9055' move completed: OK
I20240502 07:59:03.496512 25269 rebalancer_tool.cc:1136] tablet 4e202e9cf0154eb9814a7e3ad93f1e42: 'e51b10af49484aaa8f0267aebc1bf27f' -> '1aecef81733c47b3b468504934357231' move completed: OK
I20240502 07:59:03.503252 25269 rebalancer_tool.cc:1136] tablet 45925e9713cb4b748efde06f77cf3702: 'fabddb16c85944648a710ec52fc7244c' -> '1aecef81733c47b3b468504934357231' move completed: OK
I20240502 07:59:03.537839 25269 rebalancer_tool.cc:1074] tablet c5841aadb46447e685af47b70d0feff4: 'e51b10af49484aaa8f0267aebc1bf27f' -> '1aecef81733c47b3b468504934357231' move scheduled
I20240502 07:59:03.612128 25269 rebalancer_tool.cc:1074] tablet b7cdcee32c8f4cd3af30c402b8da2484: 'fabddb16c85944648a710ec52fc7244c' -> '1aecef81733c47b3b468504934357231' move scheduled
I20240502 07:59:03.669137 25269 rebalancer_tool.cc:1074] tablet dbf60f171dfe4c88840587bf0df64cac: 'e51b10af49484aaa8f0267aebc1bf27f' -> '43930dee0cd5498fa8e357a34dd71432' move scheduled
I20240502 07:59:03.701798 25269 rebalancer_tool.cc:1074] tablet b270d89546214f4d9d15193f4f8e04d5: 'fabddb16c85944648a710ec52fc7244c' -> '4b4e7eeb7f3f42119c0363a79febc54b' move scheduled
I20240502 07:59:03.729532 25269 rebalancer_tool.cc:1074] tablet e3fd481f3c78497eb629b0180c07674f: '227eec4b5f014d349661f6e0b39b9055' -> 'e51b10af49484aaa8f0267aebc1bf27f' move scheduled
I20240502 07:59:03.762895 25269 rebalancer_tool.cc:1074] tablet f7828cb273c64934853b2cd38368402f: '1aecef81733c47b3b468504934357231' -> 'e51b10af49484aaa8f0267aebc1bf27f' move scheduled
I20240502 07:59:03.796692 25269 rebalancer_tool.cc:1074] tablet af4d275b38ab49e7ac63e2fb9ffd8c07: 'f00f3a1876444ad2bf9c919068ab8014' -> 'fabddb16c85944648a710ec52fc7244c' move scheduled
I20240502 07:59:03.823256 25269 rebalancer_tool.cc:1074] tablet cc97164c8cfe4f91afabcc606d9821ef: '43930dee0cd5498fa8e357a34dd71432' -> 'e51b10af49484aaa8f0267aebc1bf27f' move scheduled
I20240502 07:59:03.852780 25269 rebalancer_tool.cc:1074] tablet 9f44c905bfaa4db093a1b39b6c21e03c: '1aecef81733c47b3b468504934357231' -> '4b4e7eeb7f3f42119c0363a79febc54b' move scheduled
I20240502 07:59:03.884773 25269 rebalancer_tool.cc:1074] tablet 288b33b7a5f84af59bde5fae34cbe924: '227eec4b5f014d349661f6e0b39b9055' -> 'fabddb16c85944648a710ec52fc7244c' move scheduled
I20240502 07:59:07.988389 25269 rebalancer_tool.cc:1136] tablet cba71d2a28984e84b1950d59ec2e5a6a: '227eec4b5f014d349661f6e0b39b9055' -> 'f00f3a1876444ad2bf9c919068ab8014' move completed: OK
I20240502 07:59:07.994086 25269 rebalancer_tool.cc:1136] tablet 41b4d8a50f684c19988a4cf1ee40921c: 'fabddb16c85944648a710ec52fc7244c' -> '1aecef81733c47b3b468504934357231' move completed: OK
I20240502 07:59:08.001289 25269 rebalancer_tool.cc:1136] tablet f49fcc586e6f498e93723d97b99c077e: '43930dee0cd5498fa8e357a34dd71432' -> 'b533a7d8d97943609c6e28e3e36b78a4' move completed: OK
I20240502 07:59:08.042827 25269 rebalancer_tool.cc:1074] tablet 15f70916ad6248e2a24334020df2c55e: 'fabddb16c85944648a710ec52fc7244c' -> '227eec4b5f014d349661f6e0b39b9055' move scheduled
I20240502 07:59:08.070773 25269 rebalancer_tool.cc:1074] tablet c651077242584f57a165073de3e4a088: '1aecef81733c47b3b468504934357231' -> 'f00f3a1876444ad2bf9c919068ab8014' move scheduled
I20240502 07:59:09.255955 25269 rebalancer_tool.cc:1136] tablet f7828cb273c64934853b2cd38368402f: '1aecef81733c47b3b468504934357231' -> 'e51b10af49484aaa8f0267aebc1bf27f' move completed: OK
I20240502 07:59:09.264497 25269 rebalancer_tool.cc:1136] tablet e3fd481f3c78497eb629b0180c07674f: '227eec4b5f014d349661f6e0b39b9055' -> 'e51b10af49484aaa8f0267aebc1bf27f' move completed: OK
I20240502 07:59:09.273799 25269 rebalancer_tool.cc:1136] tablet b270d89546214f4d9d15193f4f8e04d5: 'fabddb16c85944648a710ec52fc7244c' -> '4b4e7eeb7f3f42119c0363a79febc54b' move completed: OK
I20240502 07:59:09.279356 25269 rebalancer_tool.cc:1136] tablet dbf60f171dfe4c88840587bf0df64cac: 'e51b10af49484aaa8f0267aebc1bf27f' -> '43930dee0cd5498fa8e357a34dd71432' move completed: OK
I20240502 07:59:09.285502 25269 rebalancer_tool.cc:1136] tablet b7cdcee32c8f4cd3af30c402b8da2484: 'fabddb16c85944648a710ec52fc7244c' -> '1aecef81733c47b3b468504934357231' move completed: OK
I20240502 07:59:09.977262 25269 rebalancer_tool.cc:1136] tablet 01d9a73be1fd472db39860e538b9c97e: '227eec4b5f014d349661f6e0b39b9055' -> 'b533a7d8d97943609c6e28e3e36b78a4' move completed: OK
I20240502 07:59:09.984035 25269 rebalancer_tool.cc:1136] tablet 847f3a2601d44898a14a0b35409981ad: 'f00f3a1876444ad2bf9c919068ab8014' -> '43930dee0cd5498fa8e357a34dd71432' move completed: OK
I20240502 07:59:09.993780 25269 rebalancer_tool.cc:1136] tablet 4840fa82fcb241f391c4890fb735fe9a: 'b533a7d8d97943609c6e28e3e36b78a4' -> '4b4e7eeb7f3f42119c0363a79febc54b' move completed: OK
I20240502 07:59:09.999964 25269 rebalancer_tool.cc:1136] tablet 288b33b7a5f84af59bde5fae34cbe924: '227eec4b5f014d349661f6e0b39b9055' -> 'fabddb16c85944648a710ec52fc7244c' move completed: OK
I20240502 07:59:10.006304 25269 rebalancer_tool.cc:1136] tablet 17421eec61584fbcac13f441922269bc: 'f00f3a1876444ad2bf9c919068ab8014' -> '227eec4b5f014d349661f6e0b39b9055' move completed: OK
I20240502 07:59:10.037784 25269 rebalancer_tool.cc:1074] tablet a0f577dafc0b4846a1a9e4dc54a5a687: 'fabddb16c85944648a710ec52fc7244c' -> '227eec4b5f014d349661f6e0b39b9055' move scheduled
I20240502 07:59:10.085513 25269 rebalancer_tool.cc:1074] tablet f1890c9d5c664106bacc97d440c9cbc2: 'e51b10af49484aaa8f0267aebc1bf27f' -> '43930dee0cd5498fa8e357a34dd71432' move scheduled
I20240502 07:59:10.134078 25269 rebalancer_tool.cc:1074] tablet 895f9a31785d4558acd3b64904a645ca: 'fabddb16c85944648a710ec52fc7244c' -> '227eec4b5f014d349661f6e0b39b9055' move scheduled
I20240502 07:59:10.166386 25269 rebalancer_tool.cc:1074] tablet 61690f2a278745aeace757c233e75b25: '1aecef81733c47b3b468504934357231' -> 'f00f3a1876444ad2bf9c919068ab8014' move scheduled
I20240502 07:59:10.196734 25269 rebalancer_tool.cc:1074] tablet 6c958599492f48b7a8e4946640073d5c: '4b4e7eeb7f3f42119c0363a79febc54b' -> 'e51b10af49484aaa8f0267aebc1bf27f' move scheduled
I20240502 07:59:10.228641 25269 rebalancer_tool.cc:1074] tablet 2241a741bf85433b9fea512e9c530416: 'fabddb16c85944648a710ec52fc7244c' -> '43930dee0cd5498fa8e357a34dd71432' move scheduled
I20240502 07:59:10.267879 25269 rebalancer_tool.cc:1074] tablet 8477b961ce384edea078851122e196e9: 'f00f3a1876444ad2bf9c919068ab8014' -> '4b4e7eeb7f3f42119c0363a79febc54b' move scheduled
I20240502 07:59:12.274240 25269 rebalancer_tool.cc:1136] tablet 9f44c905bfaa4db093a1b39b6c21e03c: '1aecef81733c47b3b468504934357231' -> '4b4e7eeb7f3f42119c0363a79febc54b' move completed: OK
I20240502 07:59:12.280668 25269 rebalancer_tool.cc:1136] tablet cc97164c8cfe4f91afabcc606d9821ef: '43930dee0cd5498fa8e357a34dd71432' -> 'e51b10af49484aaa8f0267aebc1bf27f' move completed: OK
I20240502 07:59:12.287717 25269 rebalancer_tool.cc:1136] tablet af4d275b38ab49e7ac63e2fb9ffd8c07: 'f00f3a1876444ad2bf9c919068ab8014' -> 'fabddb16c85944648a710ec52fc7244c' move completed: OK
I20240502 07:59:12.607915 25269 rebalancer_tool.cc:1136] tablet c5841aadb46447e685af47b70d0feff4: 'e51b10af49484aaa8f0267aebc1bf27f' -> '1aecef81733c47b3b468504934357231' move completed: OK
I20240502 07:59:12.870023 25269 rebalancer_tool.cc:1074] tablet a0430566f5ae4ffabcf15bb8b8af6bb5: 'e51b10af49484aaa8f0267aebc1bf27f' -> '4b4e7eeb7f3f42119c0363a79febc54b' move scheduled
I20240502 07:59:13.899238 25269 rebalancer_tool.cc:1136] tablet 6c958599492f48b7a8e4946640073d5c: '4b4e7eeb7f3f42119c0363a79febc54b' -> 'e51b10af49484aaa8f0267aebc1bf27f' move completed: OK
I20240502 07:59:13.929267 25269 rebalancer_tool.cc:1136] tablet 895f9a31785d4558acd3b64904a645ca: 'fabddb16c85944648a710ec52fc7244c' -> '227eec4b5f014d349661f6e0b39b9055' move completed: OK
I20240502 07:59:13.939010 25269 rebalancer_tool.cc:1136] tablet f1890c9d5c664106bacc97d440c9cbc2: 'e51b10af49484aaa8f0267aebc1bf27f' -> '43930dee0cd5498fa8e357a34dd71432' move completed: OK
I20240502 07:59:13.945657 25269 rebalancer_tool.cc:1136] tablet a0f577dafc0b4846a1a9e4dc54a5a687: 'fabddb16c85944648a710ec52fc7244c' -> '227eec4b5f014d349661f6e0b39b9055' move completed: OK
I20240502 07:59:13.952140 25269 rebalancer_tool.cc:1136] tablet 15f70916ad6248e2a24334020df2c55e: 'fabddb16c85944648a710ec52fc7244c' -> '227eec4b5f014d349661f6e0b39b9055' move completed: OK
I20240502 07:59:13.958773 25269 rebalancer_tool.cc:1136] tablet 61690f2a278745aeace757c233e75b25: '1aecef81733c47b3b468504934357231' -> 'f00f3a1876444ad2bf9c919068ab8014' move completed: OK
I20240502 07:59:13.965257 25269 rebalancer_tool.cc:1136] tablet c651077242584f57a165073de3e4a088: '1aecef81733c47b3b468504934357231' -> 'f00f3a1876444ad2bf9c919068ab8014' move completed: OK
I20240502 07:59:13.992909 25269 rebalancer_tool.cc:1074] tablet f5f0690b92c140c49fe10ff46e0136df: 'e51b10af49484aaa8f0267aebc1bf27f' -> '4b4e7eeb7f3f42119c0363a79febc54b' move scheduled
I20240502 07:59:14.720247 25269 rebalancer_tool.cc:1136] tablet 8477b961ce384edea078851122e196e9: 'f00f3a1876444ad2bf9c919068ab8014' -> '4b4e7eeb7f3f42119c0363a79febc54b' move completed: OK
I20240502 07:59:14.726349 25269 rebalancer_tool.cc:1136] tablet 2241a741bf85433b9fea512e9c530416: 'fabddb16c85944648a710ec52fc7244c' -> '43930dee0cd5498fa8e357a34dd71432' move completed: OK
I20240502 07:59:14.766049 25269 rebalancer_tool.cc:1074] tablet a3ffa0ad72d7493082f697a1215bdbf4: '4b4e7eeb7f3f42119c0363a79febc54b' -> 'fabddb16c85944648a710ec52fc7244c' move scheduled
I20240502 07:59:15.645403 25269 rebalancer_tool.cc:1136] tablet a0430566f5ae4ffabcf15bb8b8af6bb5: 'e51b10af49484aaa8f0267aebc1bf27f' -> '4b4e7eeb7f3f42119c0363a79febc54b' move completed: OK
I20240502 07:59:15.672356 25269 rebalancer_tool.cc:1074] tablet 423d6be23b044130b0c916f995dffced: 'e51b10af49484aaa8f0267aebc1bf27f' -> '4b4e7eeb7f3f42119c0363a79febc54b' move scheduled
I20240502 07:59:16.255017 25269 rebalancer_tool.cc:1136] tablet f5f0690b92c140c49fe10ff46e0136df: 'e51b10af49484aaa8f0267aebc1bf27f' -> '4b4e7eeb7f3f42119c0363a79febc54b' move completed: OK
I20240502 07:59:16.292809 25269 rebalancer_tool.cc:1074] tablet d61cb2b6dce1409bb30d1c68c970dc22: 'e51b10af49484aaa8f0267aebc1bf27f' -> '4b4e7eeb7f3f42119c0363a79febc54b' move scheduled
I20240502 07:59:17.277868 25269 rebalancer_tool.cc:1136] tablet a3ffa0ad72d7493082f697a1215bdbf4: '4b4e7eeb7f3f42119c0363a79febc54b' -> 'fabddb16c85944648a710ec52fc7244c' move completed: OK
E20240502 07:59:17.603303 32309 ksck_remote.cc:630] unable to open table rebalancer_dynamic_table_new_name: Not found: the table does not exist: table_name: "rebalancer_dynamic_table_new_name"
I20240502 07:59:17.624979 25269 rebalancer_tool.cc:1136] tablet 423d6be23b044130b0c916f995dffced: 'e51b10af49484aaa8f0267aebc1bf27f' -> '4b4e7eeb7f3f42119c0363a79febc54b' move completed: OK
I20240502 07:59:18.870450 25269 rebalancer_tool.cc:1136] tablet d61cb2b6dce1409bb30d1c68c970dc22: 'e51b10af49484aaa8f0267aebc1bf27f' -> '4b4e7eeb7f3f42119c0363a79febc54b' move completed: OK
W20240502 07:59:19.236173 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 1ec511fc66814cefa73a9b7acff4eea9: could not find any suitable replica to move from server e51b10af49484aaa8f0267aebc1bf27f to server 43930dee0cd5498fa8e357a34dd71432
W20240502 07:59:19.236496 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table ed003ae80ddc4a70b20295b76fd5ee84: could not find any suitable replica to move from server 43930dee0cd5498fa8e357a34dd71432 to server fabddb16c85944648a710ec52fc7244c
W20240502 07:59:19.236583 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table e0a920fd5961477b92891b52ec936746: could not find any suitable replica to move from server 227eec4b5f014d349661f6e0b39b9055 to server fabddb16c85944648a710ec52fc7244c
W20240502 07:59:19.236793 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 1f887443a9c44591b3d6fac48282f904: could not find any suitable replica to move from server 4b4e7eeb7f3f42119c0363a79febc54b to server 227eec4b5f014d349661f6e0b39b9055
W20240502 07:59:19.236845 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 9e1975d2adc44bd9979e036fbd9b62ca: could not find any suitable replica to move from server 227eec4b5f014d349661f6e0b39b9055 to server e51b10af49484aaa8f0267aebc1bf27f
W20240502 07:59:19.236896 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 8d8a3547fe4849369ff81d0455ed07e5: could not find any suitable replica to move from server 1aecef81733c47b3b468504934357231 to server f00f3a1876444ad2bf9c919068ab8014
W20240502 07:59:19.236977 25269 rebalancer_tool.cc:1205] No replica could be moved this iteration: Not found: table 8bbeaa9f10a347fe87e227b0112cc204: could not find any suitable replica to move from server e51b10af49484aaa8f0267aebc1bf27f to server 227eec4b5f014d349661f6e0b39b9055
I20240502 07:59:19.268963 25269 rebalancer_tool.cc:1074] tablet 461098a3f8e64fe89a8603f78dc36dd2: '227eec4b5f014d349661f6e0b39b9055' -> 'f00f3a1876444ad2bf9c919068ab8014' move scheduled
I20240502 07:59:19.300576 25269 rebalancer_tool.cc:1074] tablet fe087f9b25c84749b7cfba34800da679: '1aecef81733c47b3b468504934357231' -> 'fabddb16c85944648a710ec52fc7244c' move scheduled
I20240502 07:59:19.332669 25269 rebalancer_tool.cc:1074] tablet 895f9a31785d4558acd3b64904a645ca: 'e51b10af49484aaa8f0267aebc1bf27f' -> '43930dee0cd5498fa8e357a34dd71432' move scheduled
I20240502 07:59:19.363176 25269 rebalancer_tool.cc:1074] tablet d63f00a2d007442781d58fb938bed480: 'b533a7d8d97943609c6e28e3e36b78a4' -> '4b4e7eeb7f3f42119c0363a79febc54b' move scheduled
I20240502 07:59:19.392640 25269 rebalancer_tool.cc:1074] tablet 5034380b5c0846faa1494140d67203cc: '227eec4b5f014d349661f6e0b39b9055' -> 'f00f3a1876444ad2bf9c919068ab8014' move scheduled
I20240502 07:59:19.426048 25269 rebalancer_tool.cc:1074] tablet ff6bc97390164a37bcdaafd651c888cf: '1aecef81733c47b3b468504934357231' -> 'fabddb16c85944648a710ec52fc7244c' move scheduled
I20240502 07:59:19.468397 25269 rebalancer_tool.cc:1074] tablet 45925e9713cb4b748efde06f77cf3702: 'e51b10af49484aaa8f0267aebc1bf27f' -> '43930dee0cd5498fa8e357a34dd71432' move scheduled
I20240502 07:59:19.502322 25269 rebalancer_tool.cc:1074] tablet c5a30e1ebd7747eb911fba8437e42077: 'b533a7d8d97943609c6e28e3e36b78a4' -> '4b4e7eeb7f3f42119c0363a79febc54b' move scheduled
I20240502 07:59:19.553560 25269 rebalancer_tool.cc:1074] tablet 26b5df5842324628b2e9bb6dc1fc36f2: '227eec4b5f014d349661f6e0b39b9055' -> 'f00f3a1876444ad2bf9c919068ab8014' move scheduled
I20240502 07:59:19.600638 25269 rebalancer_tool.cc:1074] tablet f033e8bbac8445f3af64faa67c818987: '1aecef81733c47b3b468504934357231' -> 'fabddb16c85944648a710ec52fc7244c' move scheduled
I20240502 07:59:19.636147 25269 rebalancer_tool.cc:1074] tablet 72964b1c935441ca840a46ff79921cc3: 'e51b10af49484aaa8f0267aebc1bf27f' -> '4b4e7eeb7f3f42119c0363a79febc54b' move scheduled
I20240502 07:59:19.674921 25269 rebalancer_tool.cc:1074] tablet 05911d56b0344eb8a53aba5f4bf0170c: '43930dee0cd5498fa8e357a34dd71432' -> 'b533a7d8d97943609c6e28e3e36b78a4' move scheduled
I20240502 07:59:19.713158 25269 rebalancer_tool.cc:1074] tablet 2241a741bf85433b9fea512e9c530416: 'f00f3a1876444ad2bf9c919068ab8014' -> '227eec4b5f014d349661f6e0b39b9055' move scheduled
I20240502 07:59:19.747097 25269 rebalancer_tool.cc:1074] tablet 5242aa2d2d9c477994026edb0d42051c: '1aecef81733c47b3b468504934357231' -> 'e51b10af49484aaa8f0267aebc1bf27f' move scheduled
I20240502 07:59:19.775821 25269 rebalancer_tool.cc:1074] tablet 89889b6805684edeb79afd3fa3a55dea: 'fabddb16c85944648a710ec52fc7244c' -> '4b4e7eeb7f3f42119c0363a79febc54b' move scheduled
I20240502 07:59:19.817593 25269 rebalancer_tool.cc:1074] tablet 038a695352ca4da58afaeb63a7785866: '43930dee0cd5498fa8e357a34dd71432' -> 'b533a7d8d97943609c6e28e3e36b78a4' move scheduled
I20240502 07:59:19.848215 25269 rebalancer_tool.cc:1074] tablet 2ea24049cec547b0a6cb85be6033eb3d: 'f00f3a1876444ad2bf9c919068ab8014' -> '1aecef81733c47b3b468504934357231' move scheduled
I20240502 07:59:19.880404 25269 rebalancer_tool.cc:1074] tablet a7da69fb73f94ce0b8c24599c8ae4d15: '227eec4b5f014d349661f6e0b39b9055' -> 'e51b10af49484aaa8f0267aebc1bf27f' move scheduled
I20240502 07:59:19.921991 25269 rebalancer_tool.cc:1074] tablet caf2dd9c96b94566b216692a9370f16e: '4b4e7eeb7f3f42119c0363a79febc54b' -> 'fabddb16c85944648a710ec52fc7244c' move scheduled
I20240502 07:59:19.976866 25269 rebalancer_tool.cc:1074] tablet 5d342c368796426c81bede7ddc23937b: '43930dee0cd5498fa8e357a34dd71432' -> 'b533a7d8d97943609c6e28e3e36b78a4' move scheduled
W20240502 07:59:20.506542 756 negotiation.cc:337] Failed RPC negotiation. Trace:
0502 07:59:20.265303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.129.133:41661 (local address 127.0.0.1:41198)
0502 07:59:20.266860 (+ 1557us) negotiation.cc:107] Waiting for socket to connect
0502 07:59:20.266873 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0502 07:59:20.266947 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0502 07:59:20.505707 (+238760us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1496,"thread_start_us":126,"threads_started":1}
I20240502 07:59:22.442226 25269 rebalancer_tool.cc:1123] tablet 05911d56b0344eb8a53aba5f4bf0170c: '43930dee0cd5498fa8e357a34dd71432' -> 'b533a7d8d97943609c6e28e3e36b78a4' move is abandoned: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111)
I20240502 07:59:26.762192 25269 rebalancer_tool.cc:756] re-synchronizing cluster state
Illegal state: tablet server 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): unacceptable health status UNAVAILABLE
I20240502 07:59:27.070178 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 22290
W20240502 07:59:27.081442 25020 connection.cc:535] client connection to 127.15.129.129:38535 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20240502 07:59:27.082387 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 22421
W20240502 07:59:27.083159 827 negotiation.cc:337] Failed RPC negotiation. Trace:
0502 07:59:27.081974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.129.129:38535 (local address 127.15.129.135:44355)
0502 07:59:27.082358 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0502 07:59:27.082373 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0502 07:59:27.082486 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0502 07:59:27.082592 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.129.129:38535: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":304}
W20240502 07:59:27.083647 25020 consensus_peers.cc:485] T 5d342c368796426c81bede7ddc23937b P 227eec4b5f014d349661f6e0b39b9055 -> Peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): Couldn't send request to peer b533a7d8d97943609c6e28e3e36b78a4. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.129:38535: BlockingWrite error: write error: Connection reset by peer (error 104). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:27.093789 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 22552
W20240502 07:59:27.100610 25150 consensus_peers.cc:485] T 3d1511fc27c24820afaa6e41161be8bb P 43930dee0cd5498fa8e357a34dd71432 -> Peer 4b4e7eeb7f3f42119c0363a79febc54b (127.15.129.133:41661): Couldn't send request to peer 4b4e7eeb7f3f42119c0363a79febc54b. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.133:41661: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20240502 07:59:27.102619 24885 consensus_peers.cc:485] T c951fb6e89f24b24967ce84b32e7330a P 1aecef81733c47b3b468504934357231 -> Peer fabddb16c85944648a710ec52fc7244c (127.15.129.130:39625): Couldn't send request to peer fabddb16c85944648a710ec52fc7244c. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.130:39625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240502 07:59:27.104261 22701 consensus_peers.cc:485] T 038a695352ca4da58afaeb63a7785866 P f00f3a1876444ad2bf9c919068ab8014 -> Peer b533a7d8d97943609c6e28e3e36b78a4 (127.15.129.129:38535): Couldn't send request to peer b533a7d8d97943609c6e28e3e36b78a4. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.129:38535: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:27.105849 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 22683
W20240502 07:59:27.112128 24888 consensus_peers.cc:485] T d63f00a2d007442781d58fb938bed480 P 1aecef81733c47b3b468504934357231 -> Peer e51b10af49484aaa8f0267aebc1bf27f (127.15.129.131:43703): Couldn't send request to peer e51b10af49484aaa8f0267aebc1bf27f. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.131:43703: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:27.114924 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 24870
W20240502 07:59:27.122839 25151 connection.cc:535] client connection to 127.15.129.134:35995 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240502 07:59:27.123572 25151 consensus_peers.cc:485] T bd77571fb07242f6b311c448fce0ae03 P 43930dee0cd5498fa8e357a34dd71432 -> Peer 1aecef81733c47b3b468504934357231 (127.15.129.134:35995): Couldn't send request to peer 1aecef81733c47b3b468504934357231. Status: Network error: Client connection negotiation failed: client connection to 127.15.129.134:35995: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20240502 07:59:27.123602 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 25001
I20240502 07:59:27.130970 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 25133
I20240502 07:59:27.138342 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 22199
2024-05-02T07:59:27Z chronyd exiting
I20240502 07:59:27.156381 15878 test_util.cc:175] -----------------------------------------------
I20240502 07:59:27.156438 15878 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing_0.1714636686536370-15878-0
[ FAILED ] DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing/0, where GetParam() = 4-byte object <00-00 00-00> (49076 ms)
[----------] 1 test from DDLDuringRebalancingTest (49076 ms total)
[----------] 1 test from RebalancingDuringElectionStormTest
[ RUN ] RebalancingDuringElectionStormTest.RoundRobin/0
I20240502 07:59:27.157110 15878 test_util.cc:268] Using random seed: -76752013
I20240502 07:59:27.157871 15878 ts_itest-base.cc:112] Starting cluster with:
I20240502 07:59:27.157908 15878 ts_itest-base.cc:113] --------------
I20240502 07:59:27.157919 15878 ts_itest-base.cc:114] 8 tablet servers
I20240502 07:59:27.157930 15878 ts_itest-base.cc:115] 3 replicas per TS
I20240502 07:59:27.157955 15878 ts_itest-base.cc:116] --------------
2024-05-02T07:59:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-02T07:59:27Z Disabled control of system clock
I20240502 07:59:27.166065 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.129.190:34757
--webserver_interface=127.15.129.190
--webserver_port=0
--builtin_ntp_servers=127.15.129.148:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.129.190:34757
--tserver_unresponsive_timeout_ms=3000
--enable_range_replica_placement=false
--raft_prepare_replacement_before_eviction=false
--enable_range_replica_placement=false with env {}
W20240502 07:59:27.272797 2444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:27.273032 2444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:27.273072 2444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:27.276124 2444 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:59:27.276198 2444 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240502 07:59:27.276222 2444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:27.276240 2444 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240502 07:59:27.276263 2444 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240502 07:59:27.279960 2444 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
--enable_range_replica_placement=false
--ipki_ca_key_size=768
--master_addresses=127.15.129.190:34757
--tserver_unresponsive_timeout_ms=3000
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.129.190:34757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.15.129.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.2444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:27.280891 2444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:27.282054 2444 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:27.289577 2450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:27.290339 2444 server_base.cc:1034] running on GCE node
W20240502 07:59:27.290803 2452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:27.291538 2449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:27.292100 2444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:27.293198 2444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:27.294374 2444 hybrid_clock.cc:648] HybridClock initialized: now 1714636767294350 us; error 44 us; skew 500 ppm
I20240502 07:59:27.296438 2444 webserver.cc:460] Webserver started at http://127.15.129.190:36575/ using document root <none> and password file <none>
I20240502 07:59:27.297019 2444 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:27.297086 2444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:27.297286 2444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:27.298972 2444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "e42580ca1f324724bf31a7aba67e8e89"
format_stamp: "Formatted at 2024-05-02 07:59:27 on dist-test-slave-0hx2"
I20240502 07:59:27.299437 2444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "e42580ca1f324724bf31a7aba67e8e89"
format_stamp: "Formatted at 2024-05-02 07:59:27 on dist-test-slave-0hx2"
I20240502 07:59:27.302932 2444 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20240502 07:59:27.305141 2458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:27.306164 2444 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:27.306282 2444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
uuid: "e42580ca1f324724bf31a7aba67e8e89"
format_stamp: "Formatted at 2024-05-02 07:59:27 on dist-test-slave-0hx2"
I20240502 07:59:27.306368 2444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:27.316825 2444 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:27.317411 2444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:27.317557 2444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:27.323999 2510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.190:34757 every 8 connection(s)
I20240502 07:59:27.323997 2444 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.190:34757
I20240502 07:59:27.325006 2444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20240502 07:59:27.327836 2511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:27.331390 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 2444
I20240502 07:59:27.331512 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/master-0/wal/instance
I20240502 07:59:27.333230 2511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89: Bootstrap starting.
I20240502 07:59:27.335772 2511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:27.336606 2511 log.cc:824] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:27.338443 2511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89: No bootstrap required, opened a new log
I20240502 07:59:27.341224 2511 raft_consensus.cc:348] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e42580ca1f324724bf31a7aba67e8e89" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 34757 } }
I20240502 07:59:27.341463 2511 raft_consensus.cc:374] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:27.341522 2511 raft_consensus.cc:729] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e42580ca1f324724bf31a7aba67e8e89, State: Initialized, Role: FOLLOWER
I20240502 07:59:27.341885 2511 consensus_queue.cc:260] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e42580ca1f324724bf31a7aba67e8e89" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 34757 } }
I20240502 07:59:27.341993 2511 raft_consensus.cc:388] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240502 07:59:27.342036 2511 raft_consensus.cc:482] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240502 07:59:27.342113 2511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:27.342814 2511 raft_consensus.cc:504] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e42580ca1f324724bf31a7aba67e8e89" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 34757 } }
I20240502 07:59:27.343093 2511 leader_election.cc:302] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e42580ca1f324724bf31a7aba67e8e89; no voters:
I20240502 07:59:27.343348 2511 leader_election.cc:288] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240502 07:59:27.343441 2516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:27.344038 2516 raft_consensus.cc:686] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89 [term 1 LEADER]: Becoming Leader. State: Replica: e42580ca1f324724bf31a7aba67e8e89, State: Running, Role: LEADER
I20240502 07:59:27.344316 2516 consensus_queue.cc:237] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e42580ca1f324724bf31a7aba67e8e89" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 34757 } }
I20240502 07:59:27.344326 2511 sys_catalog.cc:564] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89 [sys.catalog]: configured and running, proceeding with master startup.
I20240502 07:59:27.345659 2517 sys_catalog.cc:455] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e42580ca1f324724bf31a7aba67e8e89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e42580ca1f324724bf31a7aba67e8e89" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 34757 } } }
I20240502 07:59:27.345785 2517 sys_catalog.cc:458] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89 [sys.catalog]: This master's current role is: LEADER
I20240502 07:59:27.346094 2518 sys_catalog.cc:455] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e42580ca1f324724bf31a7aba67e8e89. Latest consensus state: current_term: 1 leader_uuid: "e42580ca1f324724bf31a7aba67e8e89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e42580ca1f324724bf31a7aba67e8e89" member_type: VOTER last_known_addr { host: "127.15.129.190" port: 34757 } } }
I20240502 07:59:27.346199 2518 sys_catalog.cc:458] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89 [sys.catalog]: This master's current role is: LEADER
W20240502 07:59:27.348796 2531 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240502 07:59:27.348860 2531 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240502 07:59:27.348934 2525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240502 07:59:27.349646 2525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240502 07:59:27.354197 2525 catalog_manager.cc:1347] Generated new cluster ID: d2d57fdab6ba4dde9725566a1b77a13b
I20240502 07:59:27.354274 2525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240502 07:59:27.384413 2525 catalog_manager.cc:1370] Generated new certificate authority record
I20240502 07:59:27.385360 2525 catalog_manager.cc:1504] Loading token signing keys...
I20240502 07:59:27.391952 2525 catalog_manager.cc:5857] T 00000000000000000000000000000000 P e42580ca1f324724bf31a7aba67e8e89: Generated new TSK 0
I20240502 07:59:27.392585 2525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240502 07:59:27.396896 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.129:0
--local_ip_for_outbound_sockets=127.15.129.129
--webserver_interface=127.15.129.129
--webserver_port=0
--tserver_master_addrs=127.15.129.190:34757
--builtin_ntp_servers=127.15.129.148:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 07:59:27.503346 2535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:27.503576 2535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:27.503645 2535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:27.506713 2535 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:59:27.506803 2535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:27.506911 2535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.129
I20240502 07:59:27.510632 2535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.15.129.129
--webserver_port=0
--tserver_master_addrs=127.15.129.190:34757
--never_fsync=true
--heap_profile_path=/tmp/kudu.2535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.129
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:27.511494 2535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:27.512555 2535 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:27.520442 2543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:27.521075 2541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:27.521747 2540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:27.522557 2535 server_base.cc:1034] running on GCE node
I20240502 07:59:27.522940 2535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:27.523485 2535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:27.524649 2535 hybrid_clock.cc:648] HybridClock initialized: now 1714636767524622 us; error 38 us; skew 500 ppm
I20240502 07:59:27.526650 2535 webserver.cc:460] Webserver started at http://127.15.129.129:37797/ using document root <none> and password file <none>
I20240502 07:59:27.527215 2535 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:27.527279 2535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:27.527451 2535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:27.529127 2535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "c93a6d22180a41a686c4556afd6a3e6b"
format_stamp: "Formatted at 2024-05-02 07:59:27 on dist-test-slave-0hx2"
I20240502 07:59:27.529630 2535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "c93a6d22180a41a686c4556afd6a3e6b"
format_stamp: "Formatted at 2024-05-02 07:59:27 on dist-test-slave-0hx2"
I20240502 07:59:27.532912 2535 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20240502 07:59:27.535169 2549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:27.536130 2535 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:27.536248 2535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "c93a6d22180a41a686c4556afd6a3e6b"
format_stamp: "Formatted at 2024-05-02 07:59:27 on dist-test-slave-0hx2"
I20240502 07:59:27.536336 2535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:27.550398 2535 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:27.550953 2535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:27.551101 2535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:27.551600 2535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:27.552439 2535 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:27.552495 2535 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:27.552551 2535 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:27.552577 2535 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:27.561460 2535 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.129:46183
I20240502 07:59:27.561479 2662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.129:46183 every 8 connection(s)
I20240502 07:59:27.562399 2535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20240502 07:59:27.562683 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 2535
I20240502 07:59:27.562820 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20240502 07:59:27.565254 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.130:0
--local_ip_for_outbound_sockets=127.15.129.130
--webserver_interface=127.15.129.130
--webserver_port=0
--tserver_master_addrs=127.15.129.190:34757
--builtin_ntp_servers=127.15.129.148:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:59:27.571432 2663 heartbeater.cc:347] Connected to a master server at 127.15.129.190:34757
I20240502 07:59:27.571647 2663 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:27.572057 2663 heartbeater.cc:511] Master 127.15.129.190:34757 requested a full tablet report, sending...
I20240502 07:59:27.573612 2475 ts_manager.cc:194] Registered new tserver with Master: c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:27.575219 2475 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.129:35731
W20240502 07:59:27.670773 2666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:27.670979 2666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:27.671020 2666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:27.673924 2666 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:59:27.673981 2666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:27.674046 2666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.130
I20240502 07:59:27.677507 2666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.15.129.130
--webserver_port=0
--tserver_master_addrs=127.15.129.190:34757
--never_fsync=true
--heap_profile_path=/tmp/kudu.2666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.130
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:27.678361 2666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:27.679415 2666 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:27.687901 2672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:27.688422 2674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:27.689289 2671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:27.689913 2666 server_base.cc:1034] running on GCE node
I20240502 07:59:27.690304 2666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:27.690876 2666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:27.692045 2666 hybrid_clock.cc:648] HybridClock initialized: now 1714636767692018 us; error 40 us; skew 500 ppm
I20240502 07:59:27.693935 2666 webserver.cc:460] Webserver started at http://127.15.129.130:33689/ using document root <none> and password file <none>
I20240502 07:59:27.694453 2666 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:27.694504 2666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:27.694649 2666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:27.696231 2666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "e0a355821af5483f96aec7e128cc18e1"
format_stamp: "Formatted at 2024-05-02 07:59:27 on dist-test-slave-0hx2"
I20240502 07:59:27.696666 2666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "e0a355821af5483f96aec7e128cc18e1"
format_stamp: "Formatted at 2024-05-02 07:59:27 on dist-test-slave-0hx2"
I20240502 07:59:27.699939 2666 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 07:59:27.702121 2680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:27.703065 2666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:59:27.703177 2666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "e0a355821af5483f96aec7e128cc18e1"
format_stamp: "Formatted at 2024-05-02 07:59:27 on dist-test-slave-0hx2"
I20240502 07:59:27.703251 2666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:27.722702 2666 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:27.723260 2666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:27.723397 2666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:27.723847 2666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:27.724663 2666 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:27.724709 2666 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:27.724747 2666 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:27.724767 2666 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:27.733145 2666 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.130:41015
I20240502 07:59:27.733217 2793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.130:41015 every 8 connection(s)
I20240502 07:59:27.734045 2666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20240502 07:59:27.740914 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 2666
I20240502 07:59:27.741034 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20240502 07:59:27.742527 2794 heartbeater.cc:347] Connected to a master server at 127.15.129.190:34757
I20240502 07:59:27.742743 2794 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:27.743094 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.131:0
--local_ip_for_outbound_sockets=127.15.129.131
--webserver_interface=127.15.129.131
--webserver_port=0
--tserver_master_addrs=127.15.129.190:34757
--builtin_ntp_servers=127.15.129.148:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:59:27.743166 2794 heartbeater.cc:511] Master 127.15.129.190:34757 requested a full tablet report, sending...
I20240502 07:59:27.744045 2475 ts_manager.cc:194] Registered new tserver with Master: e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:27.744691 2475 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.130:44317
W20240502 07:59:27.849087 2797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:27.849354 2797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:27.849421 2797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:27.852411 2797 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:59:27.852468 2797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:27.852555 2797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.131
I20240502 07:59:27.856072 2797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.15.129.131
--webserver_port=0
--tserver_master_addrs=127.15.129.190:34757
--never_fsync=true
--heap_profile_path=/tmp/kudu.2797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.131
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:27.856949 2797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:27.858086 2797 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:27.866606 2803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:27.867177 2802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:27.867812 2805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:27.868583 2797 server_base.cc:1034] running on GCE node
I20240502 07:59:27.868990 2797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:27.869565 2797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:27.870733 2797 hybrid_clock.cc:648] HybridClock initialized: now 1714636767870712 us; error 34 us; skew 500 ppm
I20240502 07:59:27.872752 2797 webserver.cc:460] Webserver started at http://127.15.129.131:35149/ using document root <none> and password file <none>
I20240502 07:59:27.873402 2797 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:27.873471 2797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:27.873751 2797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:27.875615 2797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "606c439025f24e53a2feebeaf68c1681"
format_stamp: "Formatted at 2024-05-02 07:59:27 on dist-test-slave-0hx2"
I20240502 07:59:27.876123 2797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "606c439025f24e53a2feebeaf68c1681"
format_stamp: "Formatted at 2024-05-02 07:59:27 on dist-test-slave-0hx2"
I20240502 07:59:27.879541 2797 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 07:59:27.882035 2811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:27.883081 2797 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:59:27.883209 2797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "606c439025f24e53a2feebeaf68c1681"
format_stamp: "Formatted at 2024-05-02 07:59:27 on dist-test-slave-0hx2"
I20240502 07:59:27.883304 2797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:27.904886 2797 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:27.905582 2797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:27.905745 2797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:27.906325 2797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:27.907214 2797 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:27.907267 2797 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:27.907332 2797 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:27.907361 2797 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:27.916919 2797 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.131:35417
I20240502 07:59:27.916946 2924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.131:35417 every 8 connection(s)
I20240502 07:59:27.917956 2797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20240502 07:59:27.919221 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 2797
I20240502 07:59:27.919343 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20240502 07:59:27.921384 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.132:0
--local_ip_for_outbound_sockets=127.15.129.132
--webserver_interface=127.15.129.132
--webserver_port=0
--tserver_master_addrs=127.15.129.190:34757
--builtin_ntp_servers=127.15.129.148:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:59:27.927621 2925 heartbeater.cc:347] Connected to a master server at 127.15.129.190:34757
I20240502 07:59:27.927845 2925 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:27.928283 2925 heartbeater.cc:511] Master 127.15.129.190:34757 requested a full tablet report, sending...
I20240502 07:59:27.929210 2475 ts_manager.cc:194] Registered new tserver with Master: 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:27.930007 2475 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.131:51033
W20240502 07:59:28.027307 2928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:28.027513 2928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:28.027554 2928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:28.030453 2928 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:59:28.030510 2928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:28.030573 2928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.132
I20240502 07:59:28.034031 2928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.132:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.15.129.132
--webserver_port=0
--tserver_master_addrs=127.15.129.190:34757
--never_fsync=true
--heap_profile_path=/tmp/kudu.2928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.132
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:28.034871 2928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:28.035923 2928 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:28.044317 2936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:28.044735 2933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:28.045532 2934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:28.046267 2928 server_base.cc:1034] running on GCE node
I20240502 07:59:28.046733 2928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:28.047281 2928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:28.048440 2928 hybrid_clock.cc:648] HybridClock initialized: now 1714636768048420 us; error 37 us; skew 500 ppm
I20240502 07:59:28.050369 2928 webserver.cc:460] Webserver started at http://127.15.129.132:35527/ using document root <none> and password file <none>
I20240502 07:59:28.050899 2928 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:28.050949 2928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:28.051093 2928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:28.052670 2928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:28.053124 2928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:28.056418 2928 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:28.058727 2942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:28.059702 2928 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:28.059808 2928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:28.059885 2928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:28.084195 2928 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:28.084744 2928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:28.084913 2928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:28.085497 2928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:28.086320 2928 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:28.086364 2928 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:28.086405 2928 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:28.086426 2928 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:28.094869 2928 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.132:37815
I20240502 07:59:28.094902 3055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.132:37815 every 8 connection(s)
I20240502 07:59:28.095778 2928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20240502 07:59:28.097424 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 2928
I20240502 07:59:28.097554 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20240502 07:59:28.099638 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.133:0
--local_ip_for_outbound_sockets=127.15.129.133
--webserver_interface=127.15.129.133
--webserver_port=0
--tserver_master_addrs=127.15.129.190:34757
--builtin_ntp_servers=127.15.129.148:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:59:28.104235 3056 heartbeater.cc:347] Connected to a master server at 127.15.129.190:34757
I20240502 07:59:28.104455 3056 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:28.104871 3056 heartbeater.cc:511] Master 127.15.129.190:34757 requested a full tablet report, sending...
I20240502 07:59:28.105855 2475 ts_manager.cc:194] Registered new tserver with Master: f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:28.106678 2475 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.132:60723
W20240502 07:59:28.206128 3059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:28.206354 3059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:28.206419 3059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:28.209518 3059 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:59:28.209581 3059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:28.209674 3059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.133
I20240502 07:59:28.213256 3059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.133:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.15.129.133
--webserver_port=0
--tserver_master_addrs=127.15.129.190:34757
--never_fsync=true
--heap_profile_path=/tmp/kudu.3059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.133
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:28.214172 3059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:28.215291 3059 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:28.223680 3064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:28.223780 3067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:28.224426 3065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:28.225065 3059 server_base.cc:1034] running on GCE node
I20240502 07:59:28.225453 3059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:28.225970 3059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:28.227115 3059 hybrid_clock.cc:648] HybridClock initialized: now 1714636768227088 us; error 42 us; skew 500 ppm
I20240502 07:59:28.228948 3059 webserver.cc:460] Webserver started at http://127.15.129.133:44433/ using document root <none> and password file <none>
I20240502 07:59:28.229527 3059 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:28.229589 3059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:28.229776 3059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:28.231369 3059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "4507e2f07af94d7cabcbe75708efa8d6"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:28.231838 3059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "4507e2f07af94d7cabcbe75708efa8d6"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:28.235033 3059 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 07:59:28.237110 3073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:28.238070 3059 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:28.238171 3059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "4507e2f07af94d7cabcbe75708efa8d6"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:28.238255 3059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:28.267915 3059 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:28.268559 3059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:28.268720 3059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:28.269239 3059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:28.270109 3059 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:28.270162 3059 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:28.270221 3059 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:28.270247 3059 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:28.278327 3059 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.133:44673
I20240502 07:59:28.278463 3186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.133:44673 every 8 connection(s)
I20240502 07:59:28.279249 3059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 07:59:28.285626 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 3059
I20240502 07:59:28.285758 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20240502 07:59:28.288090 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.134:0
--local_ip_for_outbound_sockets=127.15.129.134
--webserver_interface=127.15.129.134
--webserver_port=0
--tserver_master_addrs=127.15.129.190:34757
--builtin_ntp_servers=127.15.129.148:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:59:28.288779 3187 heartbeater.cc:347] Connected to a master server at 127.15.129.190:34757
I20240502 07:59:28.289001 3187 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:28.289512 3187 heartbeater.cc:511] Master 127.15.129.190:34757 requested a full tablet report, sending...
I20240502 07:59:28.290490 2475 ts_manager.cc:194] Registered new tserver with Master: 4507e2f07af94d7cabcbe75708efa8d6 (127.15.129.133:44673)
I20240502 07:59:28.291044 2475 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.133:50113
W20240502 07:59:28.398725 3190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:28.398944 3190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:28.398988 3190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:28.401916 3190 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:59:28.401971 3190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:28.402035 3190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.134
I20240502 07:59:28.405488 3190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.134:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.15.129.134
--webserver_port=0
--tserver_master_addrs=127.15.129.190:34757
--never_fsync=true
--heap_profile_path=/tmp/kudu.3190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.134
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:28.406337 3190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:28.407465 3190 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:28.415360 3195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:28.416122 3196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:28.416903 3198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:28.417584 3190 server_base.cc:1034] running on GCE node
I20240502 07:59:28.418027 3190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:28.418663 3190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:28.419818 3190 hybrid_clock.cc:648] HybridClock initialized: now 1714636768419803 us; error 34 us; skew 500 ppm
I20240502 07:59:28.421834 3190 webserver.cc:460] Webserver started at http://127.15.129.134:44541/ using document root <none> and password file <none>
I20240502 07:59:28.422394 3190 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:28.422456 3190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:28.422624 3190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:28.424299 3190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "9598319af8734926bdcfcd65c7dac9f1"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:28.424769 3190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "9598319af8734926bdcfcd65c7dac9f1"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:28.427999 3190 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 07:59:28.430183 3204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:28.431157 3190 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:59:28.431262 3190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "9598319af8734926bdcfcd65c7dac9f1"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:28.431347 3190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:28.458904 3190 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:28.459569 3190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:28.459736 3190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:28.460287 3190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:28.461194 3190 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:28.461241 3190 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:28.461299 3190 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:28.461354 3190 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:28.470395 3190 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.134:44735
I20240502 07:59:28.470427 3317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.134:44735 every 8 connection(s)
I20240502 07:59:28.471326 3190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 07:59:28.474264 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 3190
I20240502 07:59:28.474370 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20240502 07:59:28.476536 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.135:0
--local_ip_for_outbound_sockets=127.15.129.135
--webserver_interface=127.15.129.135
--webserver_port=0
--tserver_master_addrs=127.15.129.190:34757
--builtin_ntp_servers=127.15.129.148:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:59:28.480525 3318 heartbeater.cc:347] Connected to a master server at 127.15.129.190:34757
I20240502 07:59:28.480749 3318 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:28.481182 3318 heartbeater.cc:511] Master 127.15.129.190:34757 requested a full tablet report, sending...
I20240502 07:59:28.482088 2475 ts_manager.cc:194] Registered new tserver with Master: 9598319af8734926bdcfcd65c7dac9f1 (127.15.129.134:44735)
I20240502 07:59:28.482611 2475 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.134:58527
I20240502 07:59:28.577562 2663 heartbeater.cc:503] Master 127.15.129.190:34757 was elected leader, sending a full tablet report...
W20240502 07:59:28.582736 3321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:28.582975 3321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:28.583046 3321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:28.586077 3321 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:59:28.586149 3321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:28.586241 3321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.135
I20240502 07:59:28.589879 3321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.135:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.15.129.135
--webserver_port=0
--tserver_master_addrs=127.15.129.190:34757
--never_fsync=true
--heap_profile_path=/tmp/kudu.3321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.135
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:28.590857 3321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:28.592031 3321 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:28.601938 3329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:28.602491 3326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:28.602754 3327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:28.603516 3321 server_base.cc:1034] running on GCE node
I20240502 07:59:28.603950 3321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:28.604568 3321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:28.605737 3321 hybrid_clock.cc:648] HybridClock initialized: now 1714636768605713 us; error 46 us; skew 500 ppm
I20240502 07:59:28.608004 3321 webserver.cc:460] Webserver started at http://127.15.129.135:45967/ using document root <none> and password file <none>
I20240502 07:59:28.608600 3321 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:28.608665 3321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:28.608850 3321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:28.610710 3321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "ab75717712534dd7b35c2e432d65533e"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:28.611203 3321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "ab75717712534dd7b35c2e432d65533e"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:28.614806 3321 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:59:28.617195 3335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:28.618257 3321 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:59:28.618372 3321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "ab75717712534dd7b35c2e432d65533e"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:28.618485 3321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:28.631950 3321 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:28.632546 3321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:28.632709 3321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:28.633219 3321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:28.634100 3321 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:28.634156 3321 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:28.634219 3321 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:28.634248 3321 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:28.643638 3321 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.135:41793
I20240502 07:59:28.643977 3448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.135:41793 every 8 connection(s)
I20240502 07:59:28.644644 3321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 07:59:28.652700 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 3321
I20240502 07:59:28.652835 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20240502 07:59:28.653195 3449 heartbeater.cc:347] Connected to a master server at 127.15.129.190:34757
I20240502 07:59:28.653468 3449 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:28.653945 3449 heartbeater.cc:511] Master 127.15.129.190:34757 requested a full tablet report, sending...
I20240502 07:59:28.654893 2475 ts_manager.cc:194] Registered new tserver with Master: ab75717712534dd7b35c2e432d65533e (127.15.129.135:41793)
I20240502 07:59:28.655237 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.136:0
--local_ip_for_outbound_sockets=127.15.129.136
--webserver_interface=127.15.129.136
--webserver_port=0
--tserver_master_addrs=127.15.129.190:34757
--builtin_ntp_servers=127.15.129.148:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:59:28.655630 2475 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.135:46787
I20240502 07:59:28.746637 2794 heartbeater.cc:503] Master 127.15.129.190:34757 was elected leader, sending a full tablet report...
W20240502 07:59:28.765573 3452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:28.765802 3452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:28.765846 3452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:28.768781 3452 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:59:28.768839 3452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:28.768904 3452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.136
I20240502 07:59:28.772444 3452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.136:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.15.129.136
--webserver_port=0
--tserver_master_addrs=127.15.129.190:34757
--never_fsync=true
--heap_profile_path=/tmp/kudu.3452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.136
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:28.773272 3452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:28.774420 3452 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:28.783280 3458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:28.783485 3460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:28.784026 3457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:28.784894 3452 server_base.cc:1034] running on GCE node
I20240502 07:59:28.785302 3452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:28.785938 3452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:28.787111 3452 hybrid_clock.cc:648] HybridClock initialized: now 1714636768787099 us; error 28 us; skew 500 ppm
I20240502 07:59:28.789108 3452 webserver.cc:460] Webserver started at http://127.15.129.136:42365/ using document root <none> and password file <none>
I20240502 07:59:28.789716 3452 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:28.789783 3452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:28.789983 3452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240502 07:59:28.791697 3452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/instance:
uuid: "8d458f64f2f34b84bb00eea7d8d2d35a"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:28.792174 3452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/instance:
uuid: "8d458f64f2f34b84bb00eea7d8d2d35a"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:28.795449 3452 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20240502 07:59:28.797585 3466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:28.798532 3452 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20240502 07:59:28.798633 3452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "8d458f64f2f34b84bb00eea7d8d2d35a"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:28.798714 3452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:28.807495 3452 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:28.807984 3452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:28.808136 3452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:28.808596 3452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:28.809496 3452 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:28.809546 3452 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:28.809600 3452 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:28.809630 3452 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:28.817761 3452 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.136:41115
I20240502 07:59:28.817791 3579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.136:41115 every 8 connection(s)
I20240502 07:59:28.818626 3452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 07:59:28.821470 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 3452
I20240502 07:59:28.821621 15878 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal/instance
I20240502 07:59:28.829767 3580 heartbeater.cc:347] Connected to a master server at 127.15.129.190:34757
I20240502 07:59:28.829967 3580 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:28.830401 3580 heartbeater.cc:511] Master 127.15.129.190:34757 requested a full tablet report, sending...
I20240502 07:59:28.831203 2475 ts_manager.cc:194] Registered new tserver with Master: 8d458f64f2f34b84bb00eea7d8d2d35a (127.15.129.136:41115)
I20240502 07:59:28.831705 2475 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.136:59567
I20240502 07:59:28.835861 15878 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 07:59:28.845227 15878 external_mini_cluster.cc:930] 8 TS(s) registered with all masters
I20240502 07:59:28.845300 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 3059
I20240502 07:59:28.851796 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 3190
I20240502 07:59:28.857174 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 3321
I20240502 07:59:28.862337 15878 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu with pid 3452
I20240502 07:59:28.931892 2925 heartbeater.cc:503] Master 127.15.129.190:34757 was elected leader, sending a full tablet report...
I20240502 07:59:29.108594 3056 heartbeater.cc:503] Master 127.15.129.190:34757 was elected leader, sending a full tablet report...
I20240502 07:59:32.619150 2475 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53062:
name: "rebalance_test_table_0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:32.639963 2597 tablet_service.cc:1467] Processing CreateTablet for tablet 65116bced0d2437fa9a49e0391d3d4ac (DEFAULT_TABLE table=rebalance_test_table_0 [id=0d43ab38ed664fa6bab6e1fdf5c63582]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:32.639921 2990 tablet_service.cc:1467] Processing CreateTablet for tablet 0ecfaed354fe461b8ff36a2ddfa779be (DEFAULT_TABLE table=rebalance_test_table_0 [id=0d43ab38ed664fa6bab6e1fdf5c63582]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:32.639921 2858 tablet_service.cc:1467] Processing CreateTablet for tablet 65116bced0d2437fa9a49e0391d3d4ac (DEFAULT_TABLE table=rebalance_test_table_0 [id=0d43ab38ed664fa6bab6e1fdf5c63582]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:32.640890 2858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65116bced0d2437fa9a49e0391d3d4ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:32.640890 2990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ecfaed354fe461b8ff36a2ddfa779be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:32.640929 2597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65116bced0d2437fa9a49e0391d3d4ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:32.641642 2989 tablet_service.cc:1467] Processing CreateTablet for tablet 6518d4af37a1490b88f86b3de2677a5a (DEFAULT_TABLE table=rebalance_test_table_0 [id=0d43ab38ed664fa6bab6e1fdf5c63582]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:32.641922 2989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6518d4af37a1490b88f86b3de2677a5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:32.643499 2727 tablet_service.cc:1467] Processing CreateTablet for tablet 65116bced0d2437fa9a49e0391d3d4ac (DEFAULT_TABLE table=rebalance_test_table_0 [id=0d43ab38ed664fa6bab6e1fdf5c63582]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:32.644711 2859 tablet_service.cc:1467] Processing CreateTablet for tablet 0ecfaed354fe461b8ff36a2ddfa779be (DEFAULT_TABLE table=rebalance_test_table_0 [id=0d43ab38ed664fa6bab6e1fdf5c63582]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:32.644969 2859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ecfaed354fe461b8ff36a2ddfa779be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:32.645644 2596 tablet_service.cc:1467] Processing CreateTablet for tablet 6518d4af37a1490b88f86b3de2677a5a (DEFAULT_TABLE table=rebalance_test_table_0 [id=0d43ab38ed664fa6bab6e1fdf5c63582]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:32.645910 2596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6518d4af37a1490b88f86b3de2677a5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:32.644439 2727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65116bced0d2437fa9a49e0391d3d4ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:32.648186 2728 tablet_service.cc:1467] Processing CreateTablet for tablet 0ecfaed354fe461b8ff36a2ddfa779be (DEFAULT_TABLE table=rebalance_test_table_0 [id=0d43ab38ed664fa6bab6e1fdf5c63582]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:32.648488 2728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ecfaed354fe461b8ff36a2ddfa779be. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:32.648587 3600 tablet_bootstrap.cc:492] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:32.649891 2726 tablet_service.cc:1467] Processing CreateTablet for tablet 6518d4af37a1490b88f86b3de2677a5a (DEFAULT_TABLE table=rebalance_test_table_0 [id=0d43ab38ed664fa6bab6e1fdf5c63582]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:32.650187 2726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6518d4af37a1490b88f86b3de2677a5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:32.650943 3600 tablet_bootstrap.cc:654] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:32.651962 3600 log.cc:824] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:32.654336 3600 tablet_bootstrap.cc:492] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:32.654539 3600 ts_tablet_manager.cc:1367] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20240502 07:59:32.655289 3603 tablet_bootstrap.cc:492] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:32.657824 3603 tablet_bootstrap.cc:654] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:32.657883 3601 tablet_bootstrap.cc:492] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:32.658432 3604 tablet_bootstrap.cc:492] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:32.658828 3603 log.cc:824] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:32.658702 3600 raft_consensus.cc:348] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:32.658921 3600 raft_consensus.cc:374] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:32.658991 3600 raft_consensus.cc:729] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:32.659463 3600 consensus_queue.cc:260] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:32.660058 3601 tablet_bootstrap.cc:654] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:32.660352 3604 tablet_bootstrap.cc:654] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:32.660358 3600 ts_tablet_manager.cc:1398] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20240502 07:59:32.660714 3600 tablet_bootstrap.cc:492] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:32.660851 3603 tablet_bootstrap.cc:492] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:32.660821 3601 log.cc:824] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:32.661003 3603 ts_tablet_manager.cc:1367] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20240502 07:59:32.661054 3604 log.cc:824] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef: Log is configured to *not* fsync() on all Append() calls
I20240502 07:59:32.661662 3600 tablet_bootstrap.cc:654] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:32.662789 3600 tablet_bootstrap.cc:492] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:32.662873 3600 ts_tablet_manager.cc:1367] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:32.663372 3600 raft_consensus.cc:348] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:32.663527 3600 raft_consensus.cc:374] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:32.663561 3600 raft_consensus.cc:729] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:32.663724 3600 consensus_queue.cc:260] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:32.663882 3604 tablet_bootstrap.cc:492] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:32.663980 3600 ts_tablet_manager.cc:1398] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:32.664034 3604 ts_tablet_manager.cc:1367] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20240502 07:59:32.664703 3603 raft_consensus.cc:348] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:32.664902 3603 raft_consensus.cc:374] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:32.664966 3603 raft_consensus.cc:729] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:32.665616 3603 consensus_queue.cc:260] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:32.665856 3601 tablet_bootstrap.cc:492] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:32.666061 3601 ts_tablet_manager.cc:1367] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20240502 07:59:32.666491 3603 ts_tablet_manager.cc:1398] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 07:59:32.666807 3603 tablet_bootstrap.cc:492] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:32.667691 3603 tablet_bootstrap.cc:654] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:32.668920 3603 tablet_bootstrap.cc:492] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:32.668995 3603 ts_tablet_manager.cc:1367] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:32.669060 3604 raft_consensus.cc:348] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:32.669277 3604 raft_consensus.cc:374] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:32.669375 3604 raft_consensus.cc:729] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:32.669523 3603 raft_consensus.cc:348] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:32.669613 3603 raft_consensus.cc:374] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:32.669656 3603 raft_consensus.cc:729] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:32.669790 3603 consensus_queue.cc:260] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:32.669793 3604 consensus_queue.cc:260] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:32.669826 3601 raft_consensus.cc:348] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:32.670001 3601 raft_consensus.cc:374] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:32.670055 3603 ts_tablet_manager.cc:1398] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:32.670066 3601 raft_consensus.cc:729] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:32.670236 3603 tablet_bootstrap.cc:492] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:32.670480 3601 consensus_queue.cc:260] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:32.670698 3604 ts_tablet_manager.cc:1398] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20240502 07:59:32.670996 3604 tablet_bootstrap.cc:492] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:32.671104 3603 tablet_bootstrap.cc:654] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:32.671247 3601 ts_tablet_manager.cc:1398] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240502 07:59:32.671888 3604 tablet_bootstrap.cc:654] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
W20240502 07:59:32.672526 2926 tablet.cc:2369] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:59:32.672369 3601 tablet_bootstrap.cc:492] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:32.673996 3601 tablet_bootstrap.cc:654] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:32.674212 3603 tablet_bootstrap.cc:492] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:32.674285 3603 ts_tablet_manager.cc:1367] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:32.674763 3603 raft_consensus.cc:348] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:32.674863 3603 raft_consensus.cc:374] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:32.674907 3603 raft_consensus.cc:729] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:32.675019 3603 consensus_queue.cc:260] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:32.675127 3604 tablet_bootstrap.cc:492] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:32.675215 3604 ts_tablet_manager.cc:1367] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:32.675295 3603 ts_tablet_manager.cc:1398] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:32.675766 3604 raft_consensus.cc:348] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:32.675874 3604 raft_consensus.cc:374] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:32.675909 3604 raft_consensus.cc:729] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:32.676059 3604 consensus_queue.cc:260] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:32.676374 3604 ts_tablet_manager.cc:1398] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:32.677057 3601 tablet_bootstrap.cc:492] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:32.677140 3601 ts_tablet_manager.cc:1367] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:32.677695 3601 raft_consensus.cc:348] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:32.677811 3601 raft_consensus.cc:374] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:32.677842 3601 raft_consensus.cc:729] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:32.677982 3601 consensus_queue.cc:260] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:32.678265 3601 ts_tablet_manager.cc:1398] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20240502 07:59:32.738823 2795 tablet.cc:2369] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20240502 07:59:32.817340 2664 tablet.cc:2369] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:59:32.832531 3613 raft_consensus.cc:482] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:32.833025 3613 raft_consensus.cc:504] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:32.834326 3613 leader_election.cc:288] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:32.838155 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681" is_pre_election: true
I20240502 07:59:32.838375 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:32.838410 2879 raft_consensus.cc:2454] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 0.
I20240502 07:59:32.838618 2748 raft_consensus.cc:2454] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 0.
I20240502 07:59:32.838863 2551 leader_election.cc:302] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:32.839089 3613 raft_consensus.cc:2781] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:32.839164 3613 raft_consensus.cc:482] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:32.839219 3613 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:32.840063 3613 raft_consensus.cc:504] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:32.840378 3613 leader_election.cc:288] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:32.840647 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:32.840654 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:32.840735 2879 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:32.840746 2748 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:32.841590 2879 raft_consensus.cc:2454] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 1.
I20240502 07:59:32.841670 2748 raft_consensus.cc:2454] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 1.
I20240502 07:59:32.841849 2551 leader_election.cc:302] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:32.841995 3613 raft_consensus.cc:2781] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:32.842200 3613 raft_consensus.cc:686] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:32.842481 3613 consensus_queue.cc:237] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:32.845198 2474 catalog_manager.cc:5484] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 0 to 1, leader changed from <none> to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 1 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
W20240502 07:59:32.849975 3057 tablet.cc:2369] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240502 07:59:32.850490 3612 raft_consensus.cc:482] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:32.850631 3612 raft_consensus.cc:504] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:32.851598 3612 leader_election.cc:288] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:32.855312 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:32.855368 2617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b" is_pre_election: true
I20240502 07:59:32.855443 2748 raft_consensus.cc:2454] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 0.
I20240502 07:59:32.855568 2617 raft_consensus.cc:2454] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 0.
I20240502 07:59:32.855794 2946 leader_election.cc:302] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:32.856034 3612 raft_consensus.cc:2781] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:32.856107 3612 raft_consensus.cc:482] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:32.856143 3612 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:32.857054 3612 raft_consensus.cc:504] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:32.857415 3612 leader_election.cc:288] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:32.857728 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:32.857746 2617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:32.857826 2748 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:32.857843 2617 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:32.858589 2748 raft_consensus.cc:2454] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 1.
I20240502 07:59:32.858862 2946 leader_election.cc:302] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:32.859010 3612 raft_consensus.cc:2781] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:32.859202 3612 raft_consensus.cc:686] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:32.859504 3612 consensus_queue.cc:237] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:32.860715 2617 raft_consensus.cc:2454] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 1.
I20240502 07:59:32.862326 2474 catalog_manager.cc:5484] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 0 to 1, leader changed from <none> to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:33.037045 3606 raft_consensus.cc:482] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:33.037220 3606 raft_consensus.cc:504] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:33.038347 3606 leader_election.cc:288] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:33.042125 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:33.042280 2748 raft_consensus.cc:2454] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 0.
I20240502 07:59:33.042243 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" is_pre_election: true
I20240502 07:59:33.042451 3010 raft_consensus.cc:2454] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 0.
I20240502 07:59:33.043038 2815 leader_election.cc:302] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:33.043293 3606 raft_consensus.cc:2781] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:33.043354 3606 raft_consensus.cc:482] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:33.043381 3606 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.044201 3606 raft_consensus.cc:504] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:33.044565 3606 leader_election.cc:288] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:33.044871 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:33.044986 2748 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.045027 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:33.045140 3010 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.045837 2748 raft_consensus.cc:2454] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 1.
I20240502 07:59:33.046101 2815 leader_election.cc:302] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:33.046254 3606 raft_consensus.cc:2781] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:33.046450 3606 raft_consensus.cc:686] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:33.046722 3010 raft_consensus.cc:2454] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 1.
I20240502 07:59:33.046752 3606 consensus_queue.cc:237] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:33.049600 2474 catalog_manager.cc:5484] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 0 to 1, leader changed from <none> to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 1 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
W20240502 07:59:33.205412 3629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:33.205652 3629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:33.208671 3629 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_0'
I20240502 07:59:33.266232 3613 consensus_queue.cc:1035] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:33.271440 3619 consensus_queue.cc:1035] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:33.307811 2748 raft_consensus.cc:1261] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:33.308741 2617 raft_consensus.cc:1261] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:33.309247 3623 consensus_queue.cc:1035] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:33.312258 3010 raft_consensus.cc:1261] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:33.315047 2746 raft_consensus.cc:1261] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:33.315368 3627 consensus_queue.cc:1035] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:33.316478 3627 consensus_queue.cc:1035] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:33.320638 3612 consensus_queue.cc:1035] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:33.327515 3667 mvcc.cc:205] Tried to move back new op lower bound from 7023152223462301696 to 7023152221634949120. Current Snapshot: MvccSnapshot[applied={T|T < 7023152223462301696}]
I20240502 07:59:33.337540 3668 mvcc.cc:205] Tried to move back new op lower bound from 7023152223499079680 to 7023152222402113536. Current Snapshot: MvccSnapshot[applied={T|T < 7023152223475851264}]
I20240502 07:59:33.337630 3671 mvcc.cc:205] Tried to move back new op lower bound from 7023152223499079680 to 7023152222402113536. Current Snapshot: MvccSnapshot[applied={T|T < 7023152223475851264}]
I20240502 07:59:33.349947 3653 mvcc.cc:205] Tried to move back new op lower bound from 7023152223482576896 to 7023152221634949120. Current Snapshot: MvccSnapshot[applied={T|T < 7023152223462301696}]
INSERT report
rows total: 2000
time total: 120.092 ms
time per row: 0.0600459 ms
I20240502 07:59:33.376771 2472 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53062:
name: "rebalance_test_table_1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:33.382225 2859 tablet_service.cc:1467] Processing CreateTablet for tablet 236a4d787879442aaff6983243f21f85 (DEFAULT_TABLE table=rebalance_test_table_1 [id=49bf7e6db93d4d4dabd48f3278b60f0d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:33.382197 2596 tablet_service.cc:1467] Processing CreateTablet for tablet 236a4d787879442aaff6983243f21f85 (DEFAULT_TABLE table=rebalance_test_table_1 [id=49bf7e6db93d4d4dabd48f3278b60f0d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:33.382510 2596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 236a4d787879442aaff6983243f21f85. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.382510 2859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 236a4d787879442aaff6983243f21f85. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.382916 2990 tablet_service.cc:1467] Processing CreateTablet for tablet 2744610fd94b4a7ca1c7787ecc958bde (DEFAULT_TABLE table=rebalance_test_table_1 [id=49bf7e6db93d4d4dabd48f3278b60f0d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:33.383217 2990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2744610fd94b4a7ca1c7787ecc958bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.383402 2989 tablet_service.cc:1467] Processing CreateTablet for tablet 236a4d787879442aaff6983243f21f85 (DEFAULT_TABLE table=rebalance_test_table_1 [id=49bf7e6db93d4d4dabd48f3278b60f0d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:33.383622 2989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 236a4d787879442aaff6983243f21f85. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.384876 3691 tablet_bootstrap.cc:492] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:33.384974 2727 tablet_service.cc:1467] Processing CreateTablet for tablet b9b6915141b94202a356e9e6ae8b71ba (DEFAULT_TABLE table=rebalance_test_table_1 [id=49bf7e6db93d4d4dabd48f3278b60f0d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:33.385231 2727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9b6915141b94202a356e9e6ae8b71ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.385401 3692 tablet_bootstrap.cc:492] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:33.385833 3691 tablet_bootstrap.cc:654] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:33.386111 3692 tablet_bootstrap.cc:654] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:33.387022 3692 tablet_bootstrap.cc:492] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:33.387105 3692 ts_tablet_manager.cc:1367] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:33.387253 2859 tablet_service.cc:1467] Processing CreateTablet for tablet 2744610fd94b4a7ca1c7787ecc958bde (DEFAULT_TABLE table=rebalance_test_table_1 [id=49bf7e6db93d4d4dabd48f3278b60f0d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:33.387292 2858 tablet_service.cc:1467] Processing CreateTablet for tablet b9b6915141b94202a356e9e6ae8b71ba (DEFAULT_TABLE table=rebalance_test_table_1 [id=49bf7e6db93d4d4dabd48f3278b60f0d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:33.387463 2859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2744610fd94b4a7ca1c7787ecc958bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.387571 3692 raft_consensus.cc:348] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.387666 3692 raft_consensus.cc:374] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:33.387717 3692 raft_consensus.cc:729] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:33.387729 2858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9b6915141b94202a356e9e6ae8b71ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.387840 3692 consensus_queue.cc:260] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.387889 2596 tablet_service.cc:1467] Processing CreateTablet for tablet 2744610fd94b4a7ca1c7787ecc958bde (DEFAULT_TABLE table=rebalance_test_table_1 [id=49bf7e6db93d4d4dabd48f3278b60f0d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:33.388087 3692 ts_tablet_manager.cc:1398] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:33.388129 2596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2744610fd94b4a7ca1c7787ecc958bde. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.388254 3692 tablet_bootstrap.cc:492] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:33.389490 3692 tablet_bootstrap.cc:654] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:33.389689 2597 tablet_service.cc:1467] Processing CreateTablet for tablet b9b6915141b94202a356e9e6ae8b71ba (DEFAULT_TABLE table=rebalance_test_table_1 [id=49bf7e6db93d4d4dabd48f3278b60f0d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:33.389925 2597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9b6915141b94202a356e9e6ae8b71ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.390956 3692 tablet_bootstrap.cc:492] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:33.391050 3692 ts_tablet_manager.cc:1367] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 07:59:33.391407 3691 tablet_bootstrap.cc:492] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:33.391474 3692 raft_consensus.cc:348] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.391584 3692 raft_consensus.cc:374] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:33.391638 3692 raft_consensus.cc:729] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:33.391546 3691 ts_tablet_manager.cc:1367] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20240502 07:59:33.391778 3692 consensus_queue.cc:260] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.392060 3692 ts_tablet_manager.cc:1398] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:33.392114 3695 tablet_bootstrap.cc:492] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:33.392289 3691 raft_consensus.cc:348] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.392446 3691 raft_consensus.cc:374] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:33.392525 3691 raft_consensus.cc:729] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:33.392697 3691 consensus_queue.cc:260] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.393018 3695 tablet_bootstrap.cc:654] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:33.393071 3691 ts_tablet_manager.cc:1398] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:33.393239 3691 tablet_bootstrap.cc:492] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:33.394126 3691 tablet_bootstrap.cc:654] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:33.394148 3695 tablet_bootstrap.cc:492] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:33.394249 3695 ts_tablet_manager.cc:1367] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:33.394675 3695 raft_consensus.cc:348] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:33.394794 3695 raft_consensus.cc:374] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:33.394733 3690 tablet_bootstrap.cc:492] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:33.394832 3695 raft_consensus.cc:729] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:33.394953 3695 consensus_queue.cc:260] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:33.395231 3695 ts_tablet_manager.cc:1398] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:33.395421 3691 tablet_bootstrap.cc:492] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:33.395488 3691 ts_tablet_manager.cc:1367] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:33.395871 3691 raft_consensus.cc:348] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.395990 3691 raft_consensus.cc:374] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:33.396032 3691 raft_consensus.cc:729] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:33.396162 3691 consensus_queue.cc:260] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.396421 3691 ts_tablet_manager.cc:1398] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:33.396571 3691 tablet_bootstrap.cc:492] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:33.396926 3690 tablet_bootstrap.cc:654] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:33.397516 3691 tablet_bootstrap.cc:654] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:33.398620 3691 tablet_bootstrap.cc:492] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:33.398710 3691 ts_tablet_manager.cc:1367] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:33.399127 3691 raft_consensus.cc:348] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:33.399240 3691 raft_consensus.cc:374] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:33.399296 3691 raft_consensus.cc:729] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:33.399406 3691 consensus_queue.cc:260] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:33.399638 3690 tablet_bootstrap.cc:492] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:33.399677 3691 ts_tablet_manager.cc:1398] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:33.399734 3690 ts_tablet_manager.cc:1367] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:33.400169 3690 raft_consensus.cc:348] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.400285 3690 raft_consensus.cc:374] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:33.400318 3690 raft_consensus.cc:729] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:33.400456 3690 consensus_queue.cc:260] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.400681 3690 ts_tablet_manager.cc:1398] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:33.400866 3690 tablet_bootstrap.cc:492] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:33.401942 3690 tablet_bootstrap.cc:654] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:33.403034 3690 tablet_bootstrap.cc:492] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:33.403123 3690 ts_tablet_manager.cc:1367] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:59:33.403738 3690 raft_consensus.cc:348] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.403826 3690 raft_consensus.cc:374] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:33.403847 3690 raft_consensus.cc:729] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:33.403944 3690 consensus_queue.cc:260] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.404196 3690 ts_tablet_manager.cc:1398] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:33.404376 3690 tablet_bootstrap.cc:492] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:33.405122 3690 tablet_bootstrap.cc:654] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:33.406169 3690 tablet_bootstrap.cc:492] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:33.406255 3690 ts_tablet_manager.cc:1367] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 07:59:33.406693 3690 raft_consensus.cc:348] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:33.406790 3690 raft_consensus.cc:374] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:33.406826 3690 raft_consensus.cc:729] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:33.406955 3690 consensus_queue.cc:260] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:33.407189 3690 ts_tablet_manager.cc:1398] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:33.414295 3619 raft_consensus.cc:482] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:33.414405 3619 raft_consensus.cc:504] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:33.414806 3619 leader_election.cc:288] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:33.415202 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:33.415246 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681" is_pre_election: true
I20240502 07:59:33.415325 2746 raft_consensus.cc:2454] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 0.
I20240502 07:59:33.415416 2879 raft_consensus.cc:2454] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 0.
I20240502 07:59:33.415635 2553 leader_election.cc:302] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:33.415812 3619 raft_consensus.cc:2781] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:33.415869 3619 raft_consensus.cc:482] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:33.415913 3619 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.416702 3619 raft_consensus.cc:504] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:33.417061 3619 leader_election.cc:288] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:33.417409 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:33.417477 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:33.417527 2746 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.417603 2879 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.418314 2746 raft_consensus.cc:2454] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 1.
I20240502 07:59:33.418326 2879 raft_consensus.cc:2454] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 1.
I20240502 07:59:33.418597 2551 leader_election.cc:302] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:33.418651 3606 raft_consensus.cc:482] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:33.418722 3606 raft_consensus.cc:504] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.418874 3619 raft_consensus.cc:2781] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:33.418958 3619 raft_consensus.cc:686] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:33.419083 3619 consensus_queue.cc:237] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:33.419179 3606 leader_election.cc:288] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:33.420163 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" is_pre_election: true
I20240502 07:59:33.420315 3010 raft_consensus.cc:2454] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 0.
I20240502 07:59:33.420627 2812 leader_election.cc:302] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:33.420787 3606 raft_consensus.cc:2781] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:33.420833 3606 raft_consensus.cc:482] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:33.420858 3606 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.421109 2471 catalog_manager.cc:5484] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 0 to 1, leader changed from <none> to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 1 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:33.421609 3606 raft_consensus.cc:504] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.421942 3606 leader_election.cc:288] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:33.422283 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:33.422365 3010 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.423105 3010 raft_consensus.cc:2454] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 1.
I20240502 07:59:33.423511 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:33.423559 2617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b" is_pre_election: true
I20240502 07:59:33.423626 2616 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.424319 2812 leader_election.cc:302] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:33.424471 3606 raft_consensus.cc:2781] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:33.424543 3606 raft_consensus.cc:686] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:33.424566 2616 raft_consensus.cc:2454] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 1.
I20240502 07:59:33.424674 3606 consensus_queue.cc:237] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.426066 2471 catalog_manager.cc:5484] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 0 to 1, leader changed from <none> to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 1 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:33.522877 3612 raft_consensus.cc:482] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:33.523025 3612 raft_consensus.cc:504] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.523499 3612 leader_election.cc:288] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:33.524389 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b" is_pre_election: true
I20240502 07:59:33.524531 2616 raft_consensus.cc:2454] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 0.
I20240502 07:59:33.524854 2944 leader_election.cc:302] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:33.525027 3612 raft_consensus.cc:2781] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:33.525082 3612 raft_consensus.cc:482] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:33.525115 3612 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.525944 3612 raft_consensus.cc:504] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.526270 3612 leader_election.cc:288] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:33.526706 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:33.526841 2616 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.527199 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681" is_pre_election: true
I20240502 07:59:33.527274 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:33.527333 2879 raft_consensus.cc:2454] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 0.
I20240502 07:59:33.527406 2878 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.527884 2616 raft_consensus.cc:2454] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 1.
I20240502 07:59:33.528191 2944 leader_election.cc:302] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:33.528311 2878 raft_consensus.cc:2454] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 1.
I20240502 07:59:33.528425 3612 raft_consensus.cc:2781] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:33.528528 3612 raft_consensus.cc:686] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:33.528668 3612 consensus_queue.cc:237] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.530619 2471 catalog_manager.cc:5484] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 0 to 1, leader changed from <none> to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
W20240502 07:59:33.654616 3707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:33.654846 3707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:33.657932 3707 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_1'
I20240502 07:59:33.746459 2616 raft_consensus.cc:1261] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:33.746804 2878 raft_consensus.cc:1261] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:33.747548 3612 consensus_queue.cc:1035] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:33.750057 2878 raft_consensus.cc:1261] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:33.750392 2746 raft_consensus.cc:1261] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:33.750994 3619 consensus_queue.cc:1035] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:33.751258 3619 consensus_queue.cc:1035] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:33.751940 3612 consensus_queue.cc:1035] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:33.756150 3010 raft_consensus.cc:1261] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:33.756153 2617 raft_consensus.cc:1261] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:33.756664 3606 consensus_queue.cc:1035] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:33.757674 3627 consensus_queue.cc:1035] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 120.238 ms
time per row: 0.060119 ms
I20240502 07:59:33.824116 2475 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53062:
name: "rebalance_test_table_2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:33.831106 2990 tablet_service.cc:1467] Processing CreateTablet for tablet a5ce1f24891642c68ef1cf85f15158a5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=6ba11b9ccbff44c2afa34bae026387b6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:33.831506 2990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5ce1f24891642c68ef1cf85f15158a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.831784 2727 tablet_service.cc:1467] Processing CreateTablet for tablet a5ce1f24891642c68ef1cf85f15158a5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=6ba11b9ccbff44c2afa34bae026387b6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:33.832070 2727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5ce1f24891642c68ef1cf85f15158a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.833942 3692 tablet_bootstrap.cc:492] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:33.834340 2728 tablet_service.cc:1467] Processing CreateTablet for tablet 1f832b61170a4f2d95f3494c23e67e55 (DEFAULT_TABLE table=rebalance_test_table_2 [id=6ba11b9ccbff44c2afa34bae026387b6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:33.834561 2728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f832b61170a4f2d95f3494c23e67e55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.834558 2596 tablet_service.cc:1467] Processing CreateTablet for tablet 3fd58a48bc7f4d91b237d8ed1bc0277b (DEFAULT_TABLE table=rebalance_test_table_2 [id=6ba11b9ccbff44c2afa34bae026387b6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:33.834820 2596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fd58a48bc7f4d91b237d8ed1bc0277b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.834875 3692 tablet_bootstrap.cc:654] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:33.835990 3692 tablet_bootstrap.cc:492] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:33.836068 3692 ts_tablet_manager.cc:1367] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 07:59:33.836509 3692 raft_consensus.cc:348] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:33.836562 2726 tablet_service.cc:1467] Processing CreateTablet for tablet 3fd58a48bc7f4d91b237d8ed1bc0277b (DEFAULT_TABLE table=rebalance_test_table_2 [id=6ba11b9ccbff44c2afa34bae026387b6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:33.836619 3692 raft_consensus.cc:374] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:33.836652 3692 raft_consensus.cc:729] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:33.836769 2726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fd58a48bc7f4d91b237d8ed1bc0277b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.836776 3692 consensus_queue.cc:260] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:33.836989 2858 tablet_service.cc:1467] Processing CreateTablet for tablet 1f832b61170a4f2d95f3494c23e67e55 (DEFAULT_TABLE table=rebalance_test_table_2 [id=6ba11b9ccbff44c2afa34bae026387b6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:33.837082 3692 ts_tablet_manager.cc:1398] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 07:59:33.837250 2858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f832b61170a4f2d95f3494c23e67e55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.838742 2597 tablet_service.cc:1467] Processing CreateTablet for tablet a5ce1f24891642c68ef1cf85f15158a5 (DEFAULT_TABLE table=rebalance_test_table_2 [id=6ba11b9ccbff44c2afa34bae026387b6]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:33.838956 2597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5ce1f24891642c68ef1cf85f15158a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.839637 3691 tablet_bootstrap.cc:492] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:33.840512 3691 tablet_bootstrap.cc:654] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:33.840795 3695 tablet_bootstrap.cc:492] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:33.841670 3695 tablet_bootstrap.cc:654] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:33.841841 3690 tablet_bootstrap.cc:492] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:33.841899 3691 tablet_bootstrap.cc:492] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:33.841969 3691 ts_tablet_manager.cc:1367] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:33.842365 3691 raft_consensus.cc:348] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.842459 3691 raft_consensus.cc:374] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:33.842489 3691 raft_consensus.cc:729] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:33.842594 3691 consensus_queue.cc:260] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.842811 3691 ts_tablet_manager.cc:1398] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:33.842897 3690 tablet_bootstrap.cc:654] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:33.844550 2989 tablet_service.cc:1467] Processing CreateTablet for tablet 3fd58a48bc7f4d91b237d8ed1bc0277b (DEFAULT_TABLE table=rebalance_test_table_2 [id=6ba11b9ccbff44c2afa34bae026387b6]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:33.844820 2989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fd58a48bc7f4d91b237d8ed1bc0277b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.846015 3690 tablet_bootstrap.cc:492] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:33.846089 3690 ts_tablet_manager.cc:1367] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 07:59:33.846525 3690 raft_consensus.cc:348] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:33.846626 3690 raft_consensus.cc:374] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:33.846653 3690 raft_consensus.cc:729] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:33.846787 3692 tablet_bootstrap.cc:492] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:33.846767 3690 consensus_queue.cc:260] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:33.847045 3690 ts_tablet_manager.cc:1398] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:33.847214 3690 tablet_bootstrap.cc:492] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:33.847700 3692 tablet_bootstrap.cc:654] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:33.848090 3690 tablet_bootstrap.cc:654] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:33.849202 3690 tablet_bootstrap.cc:492] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:33.849287 3690 ts_tablet_manager.cc:1367] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 07:59:33.849764 3690 raft_consensus.cc:348] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:33.849875 3690 raft_consensus.cc:374] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:33.849918 3690 raft_consensus.cc:729] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:33.850037 3690 consensus_queue.cc:260] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:33.850302 3690 ts_tablet_manager.cc:1398] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:33.850525 3695 tablet_bootstrap.cc:492] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:33.850626 3695 ts_tablet_manager.cc:1367] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20240502 07:59:33.851037 3695 raft_consensus.cc:348] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:33.851148 3695 raft_consensus.cc:374] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:33.851176 3695 raft_consensus.cc:729] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:33.851289 3695 consensus_queue.cc:260] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:33.851518 3695 ts_tablet_manager.cc:1398] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:33.851665 3695 tablet_bootstrap.cc:492] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:33.853680 2990 tablet_service.cc:1467] Processing CreateTablet for tablet 1f832b61170a4f2d95f3494c23e67e55 (DEFAULT_TABLE table=rebalance_test_table_2 [id=6ba11b9ccbff44c2afa34bae026387b6]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:33.853930 2990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f832b61170a4f2d95f3494c23e67e55. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:33.856170 3695 tablet_bootstrap.cc:654] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:33.859874 3695 tablet_bootstrap.cc:492] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:33.859975 3695 ts_tablet_manager.cc:1367] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:59:33.860575 3695 raft_consensus.cc:348] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.860711 3695 raft_consensus.cc:374] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:33.860766 3695 raft_consensus.cc:729] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:33.860917 3695 consensus_queue.cc:260] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.861258 3695 ts_tablet_manager.cc:1398] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:33.861562 3695 tablet_bootstrap.cc:492] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:33.862931 3695 tablet_bootstrap.cc:654] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:33.864046 3692 tablet_bootstrap.cc:492] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:33.864143 3692 ts_tablet_manager.cc:1367] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.001s
I20240502 07:59:33.864197 3695 tablet_bootstrap.cc:492] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:33.864276 3695 ts_tablet_manager.cc:1367] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:33.864552 3692 raft_consensus.cc:348] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:33.864655 3692 raft_consensus.cc:374] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:33.864684 3692 raft_consensus.cc:729] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:33.864796 3692 consensus_queue.cc:260] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:33.864812 3695 raft_consensus.cc:348] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:33.864935 3695 raft_consensus.cc:374] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:33.864976 3695 raft_consensus.cc:729] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:33.865041 3692 ts_tablet_manager.cc:1398] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:33.865147 3695 consensus_queue.cc:260] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:33.865244 3692 tablet_bootstrap.cc:492] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:33.865622 3695 ts_tablet_manager.cc:1398] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:33.866333 3692 tablet_bootstrap.cc:654] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:33.868750 3692 tablet_bootstrap.cc:492] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:33.868830 3692 ts_tablet_manager.cc:1367] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:33.869136 3692 raft_consensus.cc:348] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.869218 3692 raft_consensus.cc:374] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:33.869251 3692 raft_consensus.cc:729] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:33.869870 3692 consensus_queue.cc:260] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.870131 3692 ts_tablet_manager.cc:1398] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:33.885479 3655 raft_consensus.cc:482] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:33.885594 3655 raft_consensus.cc:504] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:33.886585 3655 leader_election.cc:288] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:33.888659 3623 raft_consensus.cc:482] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:33.888751 3623 raft_consensus.cc:504] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.889184 3623 leader_election.cc:288] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:33.889514 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:33.889659 2746 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 0.
I20240502 07:59:33.889655 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681" is_pre_election: true
I20240502 07:59:33.889796 2878 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 0.
I20240502 07:59:33.889973 2946 leader_election.cc:302] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:33.890329 3623 raft_consensus.cc:2781] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:33.890393 3623 raft_consensus.cc:482] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:33.890427 3623 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.890952 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b" is_pre_election: true
I20240502 07:59:33.891095 2616 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 0.
I20240502 07:59:33.891501 2682 leader_election.cc:302] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:33.891577 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" is_pre_election: true
I20240502 07:59:33.891549 3623 raft_consensus.cc:504] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.891687 3010 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 0.
I20240502 07:59:33.891747 3655 raft_consensus.cc:2781] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:33.891803 3655 raft_consensus.cc:482] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:33.891853 3655 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.891897 3623 leader_election.cc:288] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:33.892172 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:33.892268 2746 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.892547 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:33.892654 2878 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.892889 3655 raft_consensus.cc:504] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:33.893121 2746 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 1.
I20240502 07:59:33.893265 3655 leader_election.cc:288] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 1 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:33.893491 2878 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 1.
I20240502 07:59:33.893597 2946 leader_election.cc:302] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:33.893790 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:33.893893 3010 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.893952 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:33.894062 2616 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.894068 3623 raft_consensus.cc:2781] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:33.894161 3623 raft_consensus.cc:686] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:33.894291 3623 consensus_queue.cc:237] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:33.894654 3010 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 1.
I20240502 07:59:33.895022 2616 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 1.
I20240502 07:59:33.895565 2681 leader_election.cc:302] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:33.895732 3655 raft_consensus.cc:2781] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:33.895889 3655 raft_consensus.cc:686] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 1 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:33.896005 2472 catalog_manager.cc:5484] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 0 to 1, leader changed from <none> to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:33.896203 3655 consensus_queue.cc:237] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:33.898240 2468 catalog_manager.cc:5484] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 1 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:33.960445 3655 raft_consensus.cc:482] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:33.960588 3655 raft_consensus.cc:504] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:33.961045 3655 leader_election.cc:288] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:33.961539 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b" is_pre_election: true
I20240502 07:59:33.961539 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" is_pre_election: true
I20240502 07:59:33.961671 2616 raft_consensus.cc:2454] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 0.
I20240502 07:59:33.961685 3010 raft_consensus.cc:2454] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 0.
I20240502 07:59:33.961966 2682 leader_election.cc:302] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:33.962136 3655 raft_consensus.cc:2781] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:33.962205 3655 raft_consensus.cc:482] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:33.962257 3655 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.963109 3655 raft_consensus.cc:504] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:33.963455 3655 leader_election.cc:288] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 1 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:33.963933 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:33.963933 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:33.964071 2616 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.964083 3010 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:33.965049 3010 raft_consensus.cc:2454] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 1.
I20240502 07:59:33.965049 2616 raft_consensus.cc:2454] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 1.
I20240502 07:59:33.965409 2681 leader_election.cc:302] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:33.965579 3655 raft_consensus.cc:2781] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:33.965663 3655 raft_consensus.cc:686] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 1 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:33.965835 3655 consensus_queue.cc:237] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:33.967551 2468 catalog_manager.cc:5484] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 1 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
W20240502 07:59:34.091966 3760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:34.092219 3760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:34.095374 3760 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_2'
I20240502 07:59:34.169148 2746 raft_consensus.cc:1261] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:34.170158 2878 raft_consensus.cc:1261] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:34.170614 3623 consensus_queue.cc:1035] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:34.170753 3612 consensus_queue.cc:1035] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:34.175622 2616 raft_consensus.cc:1261] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:34.176234 3655 consensus_queue.cc:1035] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:34.177596 2617 raft_consensus.cc:1261] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:34.178256 3010 raft_consensus.cc:1261] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:34.178282 3009 raft_consensus.cc:1261] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:34.178792 3757 consensus_queue.cc:1035] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:34.179092 3757 consensus_queue.cc:1035] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:34.182825 3655 consensus_queue.cc:1035] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 92.0881 ms
time per row: 0.046044 ms
I20240502 07:59:34.231086 2468 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53062:
name: "rebalance_test_table_3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:34.236708 2858 tablet_service.cc:1467] Processing CreateTablet for tablet 4648d978bccc4a65b30342ceaa7cf880 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0d9f7a1550b49e0a87bace44ee75fd8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:34.236936 2726 tablet_service.cc:1467] Processing CreateTablet for tablet 4648d978bccc4a65b30342ceaa7cf880 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0d9f7a1550b49e0a87bace44ee75fd8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:34.236995 2990 tablet_service.cc:1467] Processing CreateTablet for tablet d7b590b4226443d4899f3005840fd736 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0d9f7a1550b49e0a87bace44ee75fd8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:34.237054 2858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4648d978bccc4a65b30342ceaa7cf880. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:34.237247 2726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4648d978bccc4a65b30342ceaa7cf880. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:34.237242 2990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7b590b4226443d4899f3005840fd736. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:34.237694 2989 tablet_service.cc:1467] Processing CreateTablet for tablet ee852754568e497897b11ff5a7e2abfd (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0d9f7a1550b49e0a87bace44ee75fd8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:34.237905 2989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee852754568e497897b11ff5a7e2abfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:34.239266 3695 tablet_bootstrap.cc:492] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:34.239270 3691 tablet_bootstrap.cc:492] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:34.239295 3692 tablet_bootstrap.cc:492] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:34.240108 3692 tablet_bootstrap.cc:654] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:34.240108 3695 tablet_bootstrap.cc:654] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:34.240226 3691 tablet_bootstrap.cc:654] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:34.241341 2726 tablet_service.cc:1467] Processing CreateTablet for tablet d7b590b4226443d4899f3005840fd736 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0d9f7a1550b49e0a87bace44ee75fd8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:34.241591 2726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7b590b4226443d4899f3005840fd736. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:34.241746 3691 tablet_bootstrap.cc:492] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:34.241825 3691 ts_tablet_manager.cc:1367] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:34.242290 3691 raft_consensus.cc:348] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.242341 2859 tablet_service.cc:1467] Processing CreateTablet for tablet ee852754568e497897b11ff5a7e2abfd (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0d9f7a1550b49e0a87bace44ee75fd8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:34.242398 3691 raft_consensus.cc:374] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:34.242448 3691 raft_consensus.cc:729] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:34.242405 2858 tablet_service.cc:1467] Processing CreateTablet for tablet d7b590b4226443d4899f3005840fd736 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0d9f7a1550b49e0a87bace44ee75fd8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:34.242564 2859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee852754568e497897b11ff5a7e2abfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:34.242571 3691 consensus_queue.cc:260] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.242794 2858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7b590b4226443d4899f3005840fd736. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:34.243064 2597 tablet_service.cc:1467] Processing CreateTablet for tablet 4648d978bccc4a65b30342ceaa7cf880 (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0d9f7a1550b49e0a87bace44ee75fd8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:34.243258 3691 ts_tablet_manager.cc:1398] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:34.243327 2597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4648d978bccc4a65b30342ceaa7cf880. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:34.244308 3691 tablet_bootstrap.cc:492] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:34.245108 3692 tablet_bootstrap.cc:492] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:34.245108 3695 tablet_bootstrap.cc:492] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:34.245186 3692 ts_tablet_manager.cc:1367] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20240502 07:59:34.245185 3695 ts_tablet_manager.cc:1367] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20240502 07:59:34.245455 3690 tablet_bootstrap.cc:492] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:34.245579 2596 tablet_service.cc:1467] Processing CreateTablet for tablet ee852754568e497897b11ff5a7e2abfd (DEFAULT_TABLE table=rebalance_test_table_3 [id=d0d9f7a1550b49e0a87bace44ee75fd8]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:34.245673 3695 raft_consensus.cc:348] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.245673 3692 raft_consensus.cc:348] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.245771 2596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee852754568e497897b11ff5a7e2abfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:34.245772 3692 raft_consensus.cc:374] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:34.245771 3695 raft_consensus.cc:374] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:34.245806 3692 raft_consensus.cc:729] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:34.245806 3695 raft_consensus.cc:729] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:34.245927 3692 consensus_queue.cc:260] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.245927 3695 consensus_queue.cc:260] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.246171 3692 ts_tablet_manager.cc:1398] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:34.246192 3695 ts_tablet_manager.cc:1398] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:34.246322 3690 tablet_bootstrap.cc:654] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:34.246328 3692 tablet_bootstrap.cc:492] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:34.246961 3691 tablet_bootstrap.cc:654] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:34.247246 3692 tablet_bootstrap.cc:654] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:34.249066 3691 tablet_bootstrap.cc:492] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:34.249163 3691 ts_tablet_manager.cc:1367] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:34.249625 3691 raft_consensus.cc:348] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:34.249717 3691 raft_consensus.cc:374] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:34.249758 3691 raft_consensus.cc:729] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:34.249883 3691 consensus_queue.cc:260] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:34.250057 3695 tablet_bootstrap.cc:492] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:34.250133 3691 ts_tablet_manager.cc:1398] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:34.250272 3691 tablet_bootstrap.cc:492] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:34.250962 3695 tablet_bootstrap.cc:654] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:34.251052 3692 tablet_bootstrap.cc:492] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:34.251101 3691 tablet_bootstrap.cc:654] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:34.251128 3692 ts_tablet_manager.cc:1367] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20240502 07:59:34.251569 3692 raft_consensus.cc:348] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:34.251675 3692 raft_consensus.cc:374] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:34.251708 3692 raft_consensus.cc:729] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:34.251814 3692 consensus_queue.cc:260] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:34.252525 3690 tablet_bootstrap.cc:492] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:34.252547 3692 ts_tablet_manager.cc:1398] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:34.252621 3690 ts_tablet_manager.cc:1367] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20240502 07:59:34.253029 3691 tablet_bootstrap.cc:492] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:34.253077 3695 tablet_bootstrap.cc:492] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:34.253113 3691 ts_tablet_manager.cc:1367] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:34.253155 3695 ts_tablet_manager.cc:1367] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:34.253098 3690 raft_consensus.cc:348] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.253206 3690 raft_consensus.cc:374] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:34.253240 3690 raft_consensus.cc:729] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:34.253399 3690 consensus_queue.cc:260] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.253657 3690 ts_tablet_manager.cc:1398] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:34.253625 3695 raft_consensus.cc:348] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.253715 3695 raft_consensus.cc:374] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:34.253690 3691 raft_consensus.cc:348] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.253748 3695 raft_consensus.cc:729] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:34.253784 3691 raft_consensus.cc:374] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:34.253813 3690 tablet_bootstrap.cc:492] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:34.253816 3691 raft_consensus.cc:729] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:34.253861 3695 consensus_queue.cc:260] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.253926 3691 consensus_queue.cc:260] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.254088 3695 ts_tablet_manager.cc:1398] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:34.254155 3691 ts_tablet_manager.cc:1398] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:34.254709 3690 tablet_bootstrap.cc:654] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:34.257738 3690 tablet_bootstrap.cc:492] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:34.257884 3690 ts_tablet_manager.cc:1367] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20240502 07:59:34.258316 3690 raft_consensus.cc:348] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:34.258415 3690 raft_consensus.cc:374] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:34.258463 3690 raft_consensus.cc:729] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:34.258580 3690 consensus_queue.cc:260] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:34.258818 3690 ts_tablet_manager.cc:1398] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:34.260556 3606 raft_consensus.cc:482] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:34.260648 3606 raft_consensus.cc:504] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.261000 3606 leader_election.cc:288] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:34.261399 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" is_pre_election: true
I20240502 07:59:34.261463 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:34.261555 3010 raft_consensus.cc:2454] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 0.
I20240502 07:59:34.261579 2746 raft_consensus.cc:2454] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 0.
I20240502 07:59:34.261809 2815 leader_election.cc:302] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:34.261968 3606 raft_consensus.cc:2781] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:34.262019 3606 raft_consensus.cc:482] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:34.262058 3606 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:34.262804 3606 raft_consensus.cc:504] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.263092 3606 leader_election.cc:288] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:34.263373 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:34.263386 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:34.263476 3010 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:34.263489 2746 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:34.264240 2746 raft_consensus.cc:2454] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 1.
I20240502 07:59:34.264406 3010 raft_consensus.cc:2454] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 1.
I20240502 07:59:34.264489 2815 leader_election.cc:302] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:34.264632 3606 raft_consensus.cc:2781] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:34.264698 3606 raft_consensus.cc:686] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:34.264828 3606 consensus_queue.cc:237] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.266321 2468 catalog_manager.cc:5484] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 0 to 1, leader changed from <none> to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 1 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:34.379079 3606 raft_consensus.cc:482] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:34.379212 3606 raft_consensus.cc:504] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:34.379588 3606 leader_election.cc:288] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:34.379961 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" is_pre_election: true
I20240502 07:59:34.379967 2617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b" is_pre_election: true
I20240502 07:59:34.380088 3010 raft_consensus.cc:2454] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 0.
I20240502 07:59:34.380105 2617 raft_consensus.cc:2454] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 0.
I20240502 07:59:34.380354 2812 leader_election.cc:302] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:34.380506 3606 raft_consensus.cc:2781] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:34.380563 3606 raft_consensus.cc:482] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:34.380592 3606 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:34.381409 3606 raft_consensus.cc:504] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:34.381732 3606 leader_election.cc:288] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:34.382026 2617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:34.382042 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:34.382133 3010 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:34.382133 2617 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:34.382812 3010 raft_consensus.cc:2454] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 1.
I20240502 07:59:34.382946 2617 raft_consensus.cc:2454] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 1.
I20240502 07:59:34.383069 2812 leader_election.cc:302] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:34.383229 3606 raft_consensus.cc:2781] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:34.383324 3606 raft_consensus.cc:686] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:34.383468 3606 consensus_queue.cc:237] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:34.385036 2468 catalog_manager.cc:5484] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 0 to 1, leader changed from <none> to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 1 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:34.440593 3741 raft_consensus.cc:482] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:34.440747 3741 raft_consensus.cc:504] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.441287 3741 leader_election.cc:288] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:34.441748 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:34.441753 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681" is_pre_election: true
I20240502 07:59:34.441890 2746 raft_consensus.cc:2454] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 0.
I20240502 07:59:34.441910 2878 raft_consensus.cc:2454] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 0.
I20240502 07:59:34.442224 2551 leader_election.cc:302] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:34.442415 3741 raft_consensus.cc:2781] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:34.442479 3741 raft_consensus.cc:482] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:34.442516 3741 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:34.443305 3741 raft_consensus.cc:504] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.443646 3741 leader_election.cc:288] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:34.444000 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:34.444029 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:34.444128 2878 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:34.444145 2746 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:34.444854 2878 raft_consensus.cc:2454] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 1.
I20240502 07:59:34.444885 2746 raft_consensus.cc:2454] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 1.
I20240502 07:59:34.445178 2553 leader_election.cc:302] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:34.445375 3741 raft_consensus.cc:2781] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:34.445542 3741 raft_consensus.cc:686] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:34.445688 3741 consensus_queue.cc:237] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.447255 2468 catalog_manager.cc:5484] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 0 to 1, leader changed from <none> to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 1 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
W20240502 07:59:34.595849 3813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:34.596134 3813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:34.599305 3813 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_3'
I20240502 07:59:34.673856 2746 raft_consensus.cc:1261] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:34.673871 3010 raft_consensus.cc:1261] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:34.673892 3008 raft_consensus.cc:1261] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:34.674453 2617 raft_consensus.cc:1261] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:34.674521 3627 consensus_queue.cc:1035] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:34.674685 3606 consensus_queue.cc:1035] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:34.674814 3627 consensus_queue.cc:1035] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:34.677767 2747 raft_consensus.cc:1261] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:34.678128 3741 consensus_queue.cc:1035] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:34.680087 3627 consensus_queue.cc:1035] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:34.683739 2878 raft_consensus.cc:1261] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:34.684237 3741 consensus_queue.cc:1035] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 106.024 ms
time per row: 0.0530121 ms
I20240502 07:59:34.747531 2468 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53062:
name: "rebalance_test_table_4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:34.753945 2726 tablet_service.cc:1467] Processing CreateTablet for tablet 6dc9c49ec77547468d9d91c1bbee72d1 (DEFAULT_TABLE table=rebalance_test_table_4 [id=7cf439be5b7d4083a6bfd799f75d250e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:34.754245 2726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dc9c49ec77547468d9d91c1bbee72d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:34.754535 2728 tablet_service.cc:1467] Processing CreateTablet for tablet 6585f79fc41a4418b124a3e1331699c2 (DEFAULT_TABLE table=rebalance_test_table_4 [id=7cf439be5b7d4083a6bfd799f75d250e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:34.754734 2728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6585f79fc41a4418b124a3e1331699c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:34.756376 3853 tablet_bootstrap.cc:492] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:34.756937 2989 tablet_service.cc:1467] Processing CreateTablet for tablet 6585f79fc41a4418b124a3e1331699c2 (DEFAULT_TABLE table=rebalance_test_table_4 [id=7cf439be5b7d4083a6bfd799f75d250e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:34.756984 2990 tablet_service.cc:1467] Processing CreateTablet for tablet a62e14f50e3a41faa564c84e2a991ca1 (DEFAULT_TABLE table=rebalance_test_table_4 [id=7cf439be5b7d4083a6bfd799f75d250e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:34.757180 2989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6585f79fc41a4418b124a3e1331699c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:34.757285 3853 tablet_bootstrap.cc:654] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:34.757889 2597 tablet_service.cc:1467] Processing CreateTablet for tablet 6dc9c49ec77547468d9d91c1bbee72d1 (DEFAULT_TABLE table=rebalance_test_table_4 [id=7cf439be5b7d4083a6bfd799f75d250e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:34.758178 2597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dc9c49ec77547468d9d91c1bbee72d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:34.759608 3855 tablet_bootstrap.cc:492] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:34.760114 3856 tablet_bootstrap.cc:492] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:34.760556 3855 tablet_bootstrap.cc:654] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:34.760799 2990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a62e14f50e3a41faa564c84e2a991ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:34.761063 3856 tablet_bootstrap.cc:654] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:34.762255 3856 tablet_bootstrap.cc:492] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:34.762355 3856 ts_tablet_manager.cc:1367] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:34.763119 3856 raft_consensus.cc:348] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:34.763238 3856 raft_consensus.cc:374] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:34.763273 3856 raft_consensus.cc:729] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:34.763434 3856 consensus_queue.cc:260] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:34.763473 2858 tablet_service.cc:1467] Processing CreateTablet for tablet 6dc9c49ec77547468d9d91c1bbee72d1 (DEFAULT_TABLE table=rebalance_test_table_4 [id=7cf439be5b7d4083a6bfd799f75d250e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:34.763741 2858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dc9c49ec77547468d9d91c1bbee72d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:34.763803 3856 ts_tablet_manager.cc:1398] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20240502 07:59:34.763473 2859 tablet_service.cc:1467] Processing CreateTablet for tablet a62e14f50e3a41faa564c84e2a991ca1 (DEFAULT_TABLE table=rebalance_test_table_4 [id=7cf439be5b7d4083a6bfd799f75d250e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:34.765609 3859 tablet_bootstrap.cc:492] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:34.765666 2859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a62e14f50e3a41faa564c84e2a991ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:34.766458 3859 tablet_bootstrap.cc:654] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:34.767442 3859 tablet_bootstrap.cc:492] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:34.767526 3859 ts_tablet_manager.cc:1367] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:34.767817 2726 tablet_service.cc:1467] Processing CreateTablet for tablet a62e14f50e3a41faa564c84e2a991ca1 (DEFAULT_TABLE table=rebalance_test_table_4 [id=7cf439be5b7d4083a6bfd799f75d250e]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:34.768077 2726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a62e14f50e3a41faa564c84e2a991ca1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:34.767956 3859 raft_consensus.cc:348] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:34.768368 3859 raft_consensus.cc:374] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:34.768352 2597 tablet_service.cc:1467] Processing CreateTablet for tablet 6585f79fc41a4418b124a3e1331699c2 (DEFAULT_TABLE table=rebalance_test_table_4 [id=7cf439be5b7d4083a6bfd799f75d250e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:34.768424 3859 raft_consensus.cc:729] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:34.768582 2597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6585f79fc41a4418b124a3e1331699c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:34.768582 3859 consensus_queue.cc:260] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:34.768935 3859 ts_tablet_manager.cc:1398] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:34.769094 3859 tablet_bootstrap.cc:492] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:34.770116 3859 tablet_bootstrap.cc:654] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:34.770684 3856 tablet_bootstrap.cc:492] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:34.771407 3855 tablet_bootstrap.cc:492] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:34.771517 3855 ts_tablet_manager.cc:1367] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20240502 07:59:34.771636 3856 tablet_bootstrap.cc:654] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:34.771994 3855 raft_consensus.cc:348] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:34.772114 3855 raft_consensus.cc:374] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:34.772153 3855 raft_consensus.cc:729] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:34.772301 3855 consensus_queue.cc:260] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:34.772572 3855 ts_tablet_manager.cc:1398] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:34.772594 3853 tablet_bootstrap.cc:492] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:34.772681 3853 ts_tablet_manager.cc:1367] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.002s
I20240502 07:59:34.772750 3855 tablet_bootstrap.cc:492] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:34.773151 3853 raft_consensus.cc:348] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:34.773254 3853 raft_consensus.cc:374] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:34.773290 3853 raft_consensus.cc:729] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:34.773484 3853 consensus_queue.cc:260] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:34.773706 3855 tablet_bootstrap.cc:654] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:34.773752 3853 ts_tablet_manager.cc:1398] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:34.773931 3853 tablet_bootstrap.cc:492] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:34.774843 3853 tablet_bootstrap.cc:654] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:34.775820 3856 tablet_bootstrap.cc:492] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:34.775921 3856 ts_tablet_manager.cc:1367] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:34.776317 3855 tablet_bootstrap.cc:492] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:34.776326 3856 raft_consensus.cc:348] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:34.776397 3855 ts_tablet_manager.cc:1367] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:34.776435 3856 raft_consensus.cc:374] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:34.776482 3856 raft_consensus.cc:729] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:34.776612 3856 consensus_queue.cc:260] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:34.776823 3855 raft_consensus.cc:348] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.776895 3856 ts_tablet_manager.cc:1398] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:34.776911 3855 raft_consensus.cc:374] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:34.776949 3855 raft_consensus.cc:729] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:34.777074 3855 consensus_queue.cc:260] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.777354 3855 ts_tablet_manager.cc:1398] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:34.777848 3853 tablet_bootstrap.cc:492] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:34.777968 3853 ts_tablet_manager.cc:1367] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20240502 07:59:34.778458 3853 raft_consensus.cc:348] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:34.778578 3853 raft_consensus.cc:374] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:34.778631 3853 raft_consensus.cc:729] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:34.778766 3853 consensus_queue.cc:260] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:34.779104 3853 ts_tablet_manager.cc:1398] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20240502 07:59:34.779289 3853 tablet_bootstrap.cc:492] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:34.780262 3853 tablet_bootstrap.cc:654] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:34.781001 3859 tablet_bootstrap.cc:492] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:34.781083 3859 ts_tablet_manager.cc:1367] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20240502 07:59:34.781574 3859 raft_consensus.cc:348] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.781706 3859 raft_consensus.cc:374] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:34.781740 3859 raft_consensus.cc:729] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:34.781785 3853 tablet_bootstrap.cc:492] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:34.781898 3859 consensus_queue.cc:260] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.781858 3853 ts_tablet_manager.cc:1367] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 07:59:34.782228 3859 ts_tablet_manager.cc:1398] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:34.782631 3853 raft_consensus.cc:348] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.782738 3853 raft_consensus.cc:374] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:34.782783 3853 raft_consensus.cc:729] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:34.782912 3853 consensus_queue.cc:260] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.783176 3853 ts_tablet_manager.cc:1398] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:34.820008 3740 raft_consensus.cc:482] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:34.820139 3740 raft_consensus.cc:504] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.820685 3740 leader_election.cc:288] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:34.821098 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681" is_pre_election: true
I20240502 07:59:34.821110 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:34.821249 2747 raft_consensus.cc:2454] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 0.
I20240502 07:59:34.821247 2878 raft_consensus.cc:2454] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 0.
I20240502 07:59:34.821573 2944 leader_election.cc:302] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:34.821750 3740 raft_consensus.cc:2781] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:34.821796 3740 raft_consensus.cc:482] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:34.821823 3740 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:34.822633 3740 raft_consensus.cc:504] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.822927 3740 leader_election.cc:288] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:34.823261 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:34.823361 2878 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:34.823312 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:34.823447 2747 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:34.824160 2878 raft_consensus.cc:2454] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 1.
I20240502 07:59:34.824416 2944 leader_election.cc:302] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:34.824456 2747 raft_consensus.cc:2454] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 1.
I20240502 07:59:34.824586 3740 raft_consensus.cc:2781] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:34.824672 3740 raft_consensus.cc:686] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:34.824841 3740 consensus_queue.cc:237] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:34.826524 2475 catalog_manager.cc:5484] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 0 to 1, leader changed from <none> to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:34.900897 3811 raft_consensus.cc:482] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:34.901038 3811 raft_consensus.cc:504] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:34.901522 3811 leader_election.cc:288] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:34.901954 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681" is_pre_election: true
I20240502 07:59:34.901955 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:34.902088 2747 raft_consensus.cc:2454] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 0.
I20240502 07:59:34.902098 2878 raft_consensus.cc:2454] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 0.
I20240502 07:59:34.902439 2553 leader_election.cc:302] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:34.902607 3811 raft_consensus.cc:2781] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:34.902674 3811 raft_consensus.cc:482] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:34.902725 3811 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:34.903594 3811 raft_consensus.cc:504] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:34.903908 3811 leader_election.cc:288] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:34.904253 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:34.904340 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:34.904395 2878 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:34.904469 2747 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:34.905375 2747 raft_consensus.cc:2454] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 1.
I20240502 07:59:34.905416 2878 raft_consensus.cc:2454] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 1.
I20240502 07:59:34.905727 2551 leader_election.cc:302] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:34.905999 3811 raft_consensus.cc:2781] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:34.906102 3811 raft_consensus.cc:686] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:34.906268 3811 consensus_queue.cc:237] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:34.908192 2475 catalog_manager.cc:5484] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 0 to 1, leader changed from <none> to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 1 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:34.959522 3655 raft_consensus.cc:482] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:34.959673 3655 raft_consensus.cc:504] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:34.960254 3655 leader_election.cc:288] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:34.960613 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" is_pre_election: true
I20240502 07:59:34.960711 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b" is_pre_election: true
I20240502 07:59:34.960764 3010 raft_consensus.cc:2454] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 0.
I20240502 07:59:34.960831 2616 raft_consensus.cc:2454] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 0.
I20240502 07:59:34.961050 2681 leader_election.cc:302] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:34.961236 3655 raft_consensus.cc:2781] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:34.961292 3655 raft_consensus.cc:482] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:34.961338 3655 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:34.962160 3655 raft_consensus.cc:504] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:34.962481 3655 leader_election.cc:288] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 1 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:34.962821 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:34.962821 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:34.962936 3010 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:34.962936 2616 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:34.963855 3010 raft_consensus.cc:2454] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 1.
I20240502 07:59:34.963855 2616 raft_consensus.cc:2454] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 1.
I20240502 07:59:34.964115 2681 leader_election.cc:302] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:34.964298 3655 raft_consensus.cc:2781] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:34.964380 3655 raft_consensus.cc:686] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 1 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:34.964555 3655 consensus_queue.cc:237] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:34.966297 2475 catalog_manager.cc:5484] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 1 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
W20240502 07:59:35.114210 3869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:35.114531 3869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:35.118851 3869 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_4'
I20240502 07:59:35.210069 2747 raft_consensus.cc:1261] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:35.212261 3740 consensus_queue.cc:1035] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:35.213809 2878 raft_consensus.cc:1261] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:35.214143 3010 raft_consensus.cc:1261] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:35.214567 3757 consensus_queue.cc:1035] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:35.214915 2616 raft_consensus.cc:1261] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:35.215354 3757 consensus_queue.cc:1035] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:35.215363 3623 consensus_queue.cc:1035] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:35.217789 2879 raft_consensus.cc:1261] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:35.218703 3811 consensus_queue.cc:1035] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:35.220045 2746 raft_consensus.cc:1261] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:35.220441 3811 consensus_queue.cc:1035] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 114.005 ms
time per row: 0.0570027 ms
I20240502 07:59:35.279327 2475 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53062:
name: "rebalance_test_table_5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:35.284469 2597 tablet_service.cc:1467] Processing CreateTablet for tablet 410df7ea3ab44e3caa11b2c773d8555a (DEFAULT_TABLE table=rebalance_test_table_5 [id=09a6cc4b790646d0aec6b26ec0ac0b1c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:35.284495 2859 tablet_service.cc:1467] Processing CreateTablet for tablet 410df7ea3ab44e3caa11b2c773d8555a (DEFAULT_TABLE table=rebalance_test_table_5 [id=09a6cc4b790646d0aec6b26ec0ac0b1c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:35.284577 2990 tablet_service.cc:1467] Processing CreateTablet for tablet 410df7ea3ab44e3caa11b2c773d8555a (DEFAULT_TABLE table=rebalance_test_table_5 [id=09a6cc4b790646d0aec6b26ec0ac0b1c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:35.284783 2859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 410df7ea3ab44e3caa11b2c773d8555a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:35.284783 2597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 410df7ea3ab44e3caa11b2c773d8555a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:35.284806 2990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 410df7ea3ab44e3caa11b2c773d8555a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:35.286583 3924 tablet_bootstrap.cc:492] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:35.286893 3925 tablet_bootstrap.cc:492] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:35.287096 2726 tablet_service.cc:1467] Processing CreateTablet for tablet eb730198ac6b40c094863fbd7a57669a (DEFAULT_TABLE table=rebalance_test_table_5 [id=09a6cc4b790646d0aec6b26ec0ac0b1c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:35.287379 2726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb730198ac6b40c094863fbd7a57669a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:35.287627 3924 tablet_bootstrap.cc:654] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:35.287757 3925 tablet_bootstrap.cc:654] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:35.288753 3925 tablet_bootstrap.cc:492] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:35.288822 3924 tablet_bootstrap.cc:492] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:35.288842 3925 ts_tablet_manager.cc:1367] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:35.288909 3924 ts_tablet_manager.cc:1367] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:35.289425 3929 tablet_bootstrap.cc:492] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:35.289294 3925 raft_consensus.cc:348] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:35.289368 3924 raft_consensus.cc:348] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:35.289472 3925 raft_consensus.cc:374] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:35.289481 3924 raft_consensus.cc:374] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:35.289522 3924 raft_consensus.cc:729] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:35.289522 3925 raft_consensus.cc:729] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:35.289646 3925 consensus_queue.cc:260] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:35.289646 3924 consensus_queue.cc:260] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:35.289935 3924 ts_tablet_manager.cc:1398] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:35.290385 3929 tablet_bootstrap.cc:654] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:35.290422 3925 ts_tablet_manager.cc:1398] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:35.291115 2990 tablet_service.cc:1467] Processing CreateTablet for tablet 1862c4fc10fc43a8bf7e5d35c2fe47c5 (DEFAULT_TABLE table=rebalance_test_table_5 [id=09a6cc4b790646d0aec6b26ec0ac0b1c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:35.291357 2990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1862c4fc10fc43a8bf7e5d35c2fe47c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:35.291405 3929 tablet_bootstrap.cc:492] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:35.291482 3929 ts_tablet_manager.cc:1367] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:35.291906 3929 raft_consensus.cc:348] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:35.291996 3929 raft_consensus.cc:374] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:35.292032 3929 raft_consensus.cc:729] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:35.292148 3929 consensus_queue.cc:260] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:35.292585 3929 ts_tablet_manager.cc:1398] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:35.292688 2597 tablet_service.cc:1467] Processing CreateTablet for tablet 1862c4fc10fc43a8bf7e5d35c2fe47c5 (DEFAULT_TABLE table=rebalance_test_table_5 [id=09a6cc4b790646d0aec6b26ec0ac0b1c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:35.292908 2597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1862c4fc10fc43a8bf7e5d35c2fe47c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:35.293262 3925 tablet_bootstrap.cc:492] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:35.293989 3925 tablet_bootstrap.cc:654] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:35.294541 3924 tablet_bootstrap.cc:492] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:35.294728 2596 tablet_service.cc:1467] Processing CreateTablet for tablet eb730198ac6b40c094863fbd7a57669a (DEFAULT_TABLE table=rebalance_test_table_5 [id=09a6cc4b790646d0aec6b26ec0ac0b1c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:35.294891 2596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb730198ac6b40c094863fbd7a57669a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:35.294989 3925 tablet_bootstrap.cc:492] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:35.295078 3925 ts_tablet_manager.cc:1367] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20240502 07:59:35.295560 3924 tablet_bootstrap.cc:654] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:35.295506 3925 raft_consensus.cc:348] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:35.295614 3925 raft_consensus.cc:374] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:35.295646 3925 raft_consensus.cc:729] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:35.295786 3925 consensus_queue.cc:260] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:35.296052 3925 ts_tablet_manager.cc:1398] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:35.296437 2859 tablet_service.cc:1467] Processing CreateTablet for tablet 1862c4fc10fc43a8bf7e5d35c2fe47c5 (DEFAULT_TABLE table=rebalance_test_table_5 [id=09a6cc4b790646d0aec6b26ec0ac0b1c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:35.296581 3924 tablet_bootstrap.cc:492] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:35.296663 3924 ts_tablet_manager.cc:1367] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:35.296676 2859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1862c4fc10fc43a8bf7e5d35c2fe47c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:35.296994 3924 raft_consensus.cc:348] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:35.297078 3924 raft_consensus.cc:374] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:35.297113 3924 raft_consensus.cc:729] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:35.297250 3924 consensus_queue.cc:260] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:35.297533 3924 ts_tablet_manager.cc:1398] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:35.297680 3924 tablet_bootstrap.cc:492] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:35.298328 3924 tablet_bootstrap.cc:654] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:35.298434 2858 tablet_service.cc:1467] Processing CreateTablet for tablet eb730198ac6b40c094863fbd7a57669a (DEFAULT_TABLE table=rebalance_test_table_5 [id=09a6cc4b790646d0aec6b26ec0ac0b1c]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:35.298622 2858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb730198ac6b40c094863fbd7a57669a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:35.299252 3924 tablet_bootstrap.cc:492] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:35.299326 3924 ts_tablet_manager.cc:1367] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:35.299643 3924 raft_consensus.cc:348] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:35.299724 3924 raft_consensus.cc:374] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:35.299758 3924 raft_consensus.cc:729] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:35.299873 3924 consensus_queue.cc:260] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:35.300077 3924 ts_tablet_manager.cc:1398] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:35.301841 3926 tablet_bootstrap.cc:492] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:35.302536 3926 tablet_bootstrap.cc:654] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:35.303496 3926 tablet_bootstrap.cc:492] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:35.303571 3926 ts_tablet_manager.cc:1367] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:35.303894 3926 raft_consensus.cc:348] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:35.303977 3926 raft_consensus.cc:374] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:35.303997 3926 raft_consensus.cc:729] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:35.304098 3926 consensus_queue.cc:260] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:35.304334 3926 ts_tablet_manager.cc:1398] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:35.304481 3926 tablet_bootstrap.cc:492] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:35.305227 3926 tablet_bootstrap.cc:654] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:35.306283 3926 tablet_bootstrap.cc:492] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:35.306360 3926 ts_tablet_manager.cc:1367] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:35.306703 3926 raft_consensus.cc:348] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:35.306800 3926 raft_consensus.cc:374] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:35.306825 3926 raft_consensus.cc:729] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:35.306944 3926 consensus_queue.cc:260] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:35.307191 3926 ts_tablet_manager.cc:1398] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:35.307346 3926 tablet_bootstrap.cc:492] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:35.308249 3926 tablet_bootstrap.cc:654] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:35.309422 3926 tablet_bootstrap.cc:492] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:35.309568 3926 ts_tablet_manager.cc:1367] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:35.309964 3926 raft_consensus.cc:348] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:35.310072 3926 raft_consensus.cc:374] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:35.310097 3926 raft_consensus.cc:729] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:35.310187 3926 consensus_queue.cc:260] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:35.310424 3926 ts_tablet_manager.cc:1398] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:35.320214 3741 raft_consensus.cc:482] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:35.320366 3741 raft_consensus.cc:504] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:35.320812 3741 leader_election.cc:288] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:35.321178 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681" is_pre_election: true
I20240502 07:59:35.321352 2879 raft_consensus.cc:2454] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 0.
I20240502 07:59:35.321641 2551 leader_election.cc:302] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:35.321848 3741 raft_consensus.cc:2781] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:35.321902 3741 raft_consensus.cc:482] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:35.321942 3741 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:35.322736 3741 raft_consensus.cc:504] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:35.323083 3741 leader_election.cc:288] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:35.323395 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:35.323501 2879 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:35.324245 2879 raft_consensus.cc:2454] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 1.
I20240502 07:59:35.324574 2551 leader_election.cc:302] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:35.324771 3741 raft_consensus.cc:2781] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:35.324877 3741 raft_consensus.cc:686] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:35.325018 3741 consensus_queue.cc:237] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:35.326205 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" is_pre_election: true
I20240502 07:59:35.326262 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:35.326346 3010 raft_consensus.cc:2454] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 0.
I20240502 07:59:35.330389 2475 catalog_manager.cc:5484] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 0 to 1, leader changed from <none> to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 1 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:35.615558 3741 raft_consensus.cc:482] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:35.615687 3741 raft_consensus.cc:504] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:35.616083 3741 leader_election.cc:288] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:35.616542 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:35.616566 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681" is_pre_election: true
I20240502 07:59:35.616684 2746 raft_consensus.cc:2454] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 0.
I20240502 07:59:35.616722 2879 raft_consensus.cc:2454] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 0.
I20240502 07:59:35.616999 2553 leader_election.cc:302] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:35.617172 3741 raft_consensus.cc:2781] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:35.617233 3741 raft_consensus.cc:482] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:35.617287 3741 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:35.618155 3741 raft_consensus.cc:504] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:35.618536 3741 leader_election.cc:288] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:35.618912 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:35.618932 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:35.619024 2746 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:35.619048 2879 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:35.619788 2746 raft_consensus.cc:2454] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 1.
I20240502 07:59:35.619796 2879 raft_consensus.cc:2454] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 1.
I20240502 07:59:35.620092 2551 leader_election.cc:302] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:35.620328 3741 raft_consensus.cc:2781] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:35.620399 3741 raft_consensus.cc:686] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:35.620533 3741 consensus_queue.cc:237] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:35.622601 2475 catalog_manager.cc:5484] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 0 to 1, leader changed from <none> to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 1 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:35.664000 3843 raft_consensus.cc:482] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:35.664131 3843 raft_consensus.cc:504] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:35.664577 3843 leader_election.cc:288] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:35.664889 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" is_pre_election: true
I20240502 07:59:35.664944 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b" is_pre_election: true
I20240502 07:59:35.665015 3008 raft_consensus.cc:2454] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 0.
I20240502 07:59:35.665081 2616 raft_consensus.cc:2454] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 0.
I20240502 07:59:35.665280 2812 leader_election.cc:302] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:35.665566 3843 raft_consensus.cc:2781] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:35.665623 3843 raft_consensus.cc:482] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:35.665676 3843 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:35.666520 3843 raft_consensus.cc:504] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:35.666833 3843 leader_election.cc:288] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:35.667109 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:35.667208 3008 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:35.667174 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:35.667310 2616 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:35.668040 2616 raft_consensus.cc:2454] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 1.
I20240502 07:59:35.668165 3008 raft_consensus.cc:2454] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 1.
I20240502 07:59:35.668354 2813 leader_election.cc:302] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:35.668524 3843 raft_consensus.cc:2781] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:35.668701 3843 raft_consensus.cc:686] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:35.668848 3843 consensus_queue.cc:237] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:35.670437 2475 catalog_manager.cc:5484] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 0 to 1, leader changed from <none> to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 1 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:35.725621 3008 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:35.726948 3741 consensus_queue.cc:1035] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:35.730044 3741 consensus_queue.cc:1035] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 07:59:35.862294 3944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:35.862522 3944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:35.865578 3944 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_5'
I20240502 07:59:35.939138 2616 raft_consensus.cc:1261] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:35.939735 3843 consensus_queue.cc:1035] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:35.940608 3008 raft_consensus.cc:1261] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:35.941151 3940 consensus_queue.cc:1035] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:35.946267 2879 raft_consensus.cc:1261] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:35.946943 2746 raft_consensus.cc:1261] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:35.947492 3741 consensus_queue.cc:1035] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:35.949718 3811 consensus_queue.cc:1035] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:35.962388 3972 mvcc.cc:205] Tried to move back new op lower bound from 7023152234284273664 to 7023152232943091712. Current Snapshot: MvccSnapshot[applied={T|T < 7023152234254663680}]
I20240502 07:59:35.964334 3850 mvcc.cc:205] Tried to move back new op lower bound from 7023152234254663680 to 7023152232943091712. Current Snapshot: MvccSnapshot[applied={T|T < 7023152234254663680}]
INSERT report
rows total: 2000
time total: 97.5344 ms
time per row: 0.0487672 ms
I20240502 07:59:35.994711 3981 mvcc.cc:205] Tried to move back new op lower bound from 7023152234284273664 to 7023152232943091712. Current Snapshot: MvccSnapshot[applied={T|T < 7023152234284273664}]
I20240502 07:59:36.004506 2473 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53062:
name: "rebalance_test_table_6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:36.009809 2990 tablet_service.cc:1467] Processing CreateTablet for tablet 811167eb170b49219b6cd22d865af334 (DEFAULT_TABLE table=rebalance_test_table_6 [id=1505a2d3adf247a88406a4f50932739b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:36.009791 2726 tablet_service.cc:1467] Processing CreateTablet for tablet 811167eb170b49219b6cd22d865af334 (DEFAULT_TABLE table=rebalance_test_table_6 [id=1505a2d3adf247a88406a4f50932739b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:36.010103 2990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 811167eb170b49219b6cd22d865af334. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.010107 2726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 811167eb170b49219b6cd22d865af334. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.010767 2596 tablet_service.cc:1467] Processing CreateTablet for tablet 811167eb170b49219b6cd22d865af334 (DEFAULT_TABLE table=rebalance_test_table_6 [id=1505a2d3adf247a88406a4f50932739b]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:36.011022 2596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 811167eb170b49219b6cd22d865af334. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.011055 2989 tablet_service.cc:1467] Processing CreateTablet for tablet 40a5b1a2b986401ea3166e7a3453cc0e (DEFAULT_TABLE table=rebalance_test_table_6 [id=1505a2d3adf247a88406a4f50932739b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:36.011111 2988 tablet_service.cc:1467] Processing CreateTablet for tablet 3fb8f0e2326646b488f6d25a8264e8a7 (DEFAULT_TABLE table=rebalance_test_table_6 [id=1505a2d3adf247a88406a4f50932739b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:36.011269 2989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40a5b1a2b986401ea3166e7a3453cc0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.011592 2988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fb8f0e2326646b488f6d25a8264e8a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.011754 2858 tablet_service.cc:1467] Processing CreateTablet for tablet 40a5b1a2b986401ea3166e7a3453cc0e (DEFAULT_TABLE table=rebalance_test_table_6 [id=1505a2d3adf247a88406a4f50932739b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:36.012014 2858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40a5b1a2b986401ea3166e7a3453cc0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.012748 2597 tablet_service.cc:1467] Processing CreateTablet for tablet 3fb8f0e2326646b488f6d25a8264e8a7 (DEFAULT_TABLE table=rebalance_test_table_6 [id=1505a2d3adf247a88406a4f50932739b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:36.012928 2597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fb8f0e2326646b488f6d25a8264e8a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.013096 3997 tablet_bootstrap.cc:492] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:36.014010 3998 tablet_bootstrap.cc:492] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:36.014113 3997 tablet_bootstrap.cc:654] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.014935 3998 tablet_bootstrap.cc:654] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.015023 2728 tablet_service.cc:1467] Processing CreateTablet for tablet 40a5b1a2b986401ea3166e7a3453cc0e (DEFAULT_TABLE table=rebalance_test_table_6 [id=1505a2d3adf247a88406a4f50932739b]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:36.015252 2728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40a5b1a2b986401ea3166e7a3453cc0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.015460 3997 tablet_bootstrap.cc:492] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:36.015539 3997 ts_tablet_manager.cc:1367] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:36.015879 3998 tablet_bootstrap.cc:492] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:36.015937 3998 ts_tablet_manager.cc:1367] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:36.015934 3997 raft_consensus.cc:348] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:36.016043 3997 raft_consensus.cc:374] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.016075 3997 raft_consensus.cc:729] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.016222 3997 consensus_queue.cc:260] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:36.016273 3998 raft_consensus.cc:348] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:36.016353 3998 raft_consensus.cc:374] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.016382 3998 raft_consensus.cc:729] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.016523 3998 consensus_queue.cc:260] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:36.016760 3998 ts_tablet_manager.cc:1398] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:36.016976 3997 ts_tablet_manager.cc:1398] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:36.017128 3997 tablet_bootstrap.cc:492] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:36.017998 3997 tablet_bootstrap.cc:654] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.018293 2728 tablet_service.cc:1467] Processing CreateTablet for tablet 3fb8f0e2326646b488f6d25a8264e8a7 (DEFAULT_TABLE table=rebalance_test_table_6 [id=1505a2d3adf247a88406a4f50932739b]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:36.018522 2728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fb8f0e2326646b488f6d25a8264e8a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.019816 3997 tablet_bootstrap.cc:492] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:36.019980 3997 ts_tablet_manager.cc:1367] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:36.020427 3997 raft_consensus.cc:348] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:36.020538 3997 raft_consensus.cc:374] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.020586 3997 raft_consensus.cc:729] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.020622 3996 tablet_bootstrap.cc:492] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:36.020733 3997 consensus_queue.cc:260] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:36.021464 3996 tablet_bootstrap.cc:654] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.021456 3997 ts_tablet_manager.cc:1398] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:36.041847 3996 tablet_bootstrap.cc:492] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:36.042074 3996 ts_tablet_manager.cc:1367] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.002s
I20240502 07:59:36.042749 3996 raft_consensus.cc:348] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:36.042944 3996 raft_consensus.cc:374] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.043026 3996 raft_consensus.cc:729] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.043226 3996 consensus_queue.cc:260] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:36.043684 4003 tablet_bootstrap.cc:492] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:36.044572 3996 ts_tablet_manager.cc:1398] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:36.044668 4003 tablet_bootstrap.cc:654] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.044806 3996 tablet_bootstrap.cc:492] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:36.045809 3996 tablet_bootstrap.cc:654] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.045951 4003 tablet_bootstrap.cc:492] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:36.046042 4003 ts_tablet_manager.cc:1367] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:36.046500 4003 raft_consensus.cc:348] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:36.046625 4003 raft_consensus.cc:374] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.046663 4003 raft_consensus.cc:729] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.046888 3996 tablet_bootstrap.cc:492] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:36.046840 4003 consensus_queue.cc:260] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:36.046972 3996 ts_tablet_manager.cc:1367] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:36.047195 4003 ts_tablet_manager.cc:1398] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:36.047358 3996 raft_consensus.cc:348] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:36.047417 4003 tablet_bootstrap.cc:492] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:36.047451 3996 raft_consensus.cc:374] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.047502 3996 raft_consensus.cc:729] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.047616 3996 consensus_queue.cc:260] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:36.047883 3996 ts_tablet_manager.cc:1398] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:36.048023 3996 tablet_bootstrap.cc:492] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:36.048529 4003 tablet_bootstrap.cc:654] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.049052 3996 tablet_bootstrap.cc:654] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.050096 4003 tablet_bootstrap.cc:492] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:36.050177 3996 tablet_bootstrap.cc:492] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:36.050180 4003 ts_tablet_manager.cc:1367] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:36.050318 3996 ts_tablet_manager.cc:1367] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20240502 07:59:36.050623 4003 raft_consensus.cc:348] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:36.050733 4003 raft_consensus.cc:374] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.050779 4003 raft_consensus.cc:729] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.050736 3996 raft_consensus.cc:348] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:36.050824 3996 raft_consensus.cc:374] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.050865 3996 raft_consensus.cc:729] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.050908 4003 consensus_queue.cc:260] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:36.050969 3996 consensus_queue.cc:260] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:36.051172 4003 ts_tablet_manager.cc:1398] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:36.051223 3996 ts_tablet_manager.cc:1398] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:36.051337 4003 tablet_bootstrap.cc:492] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:36.052163 4003 tablet_bootstrap.cc:654] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.053128 3914 raft_consensus.cc:482] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:36.053201 3914 raft_consensus.cc:504] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:36.053606 3914 leader_election.cc:288] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:36.053655 4003 tablet_bootstrap.cc:492] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:36.053733 4003 ts_tablet_manager.cc:1367] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:36.053927 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:36.053948 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b" is_pre_election: true
I20240502 07:59:36.054041 2746 raft_consensus.cc:2454] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 0.
I20240502 07:59:36.054069 2616 raft_consensus.cc:2454] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 0.
I20240502 07:59:36.054096 4003 raft_consensus.cc:348] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:36.054163 4003 raft_consensus.cc:374] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.054183 4003 raft_consensus.cc:729] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.054363 2944 leader_election.cc:302] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:36.054514 3914 raft_consensus.cc:2781] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:36.054570 3914 raft_consensus.cc:482] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:36.054636 3914 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.054572 4003 consensus_queue.cc:260] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:36.054988 4003 ts_tablet_manager.cc:1398] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:36.055581 3914 raft_consensus.cc:504] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:36.055908 3914 leader_election.cc:288] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:36.056200 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:36.056248 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:36.056337 2616 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.056342 2746 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.057199 2746 raft_consensus.cc:2454] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 1.
I20240502 07:59:36.057265 2616 raft_consensus.cc:2454] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 1.
I20240502 07:59:36.057551 2946 leader_election.cc:302] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:36.057716 3914 raft_consensus.cc:2781] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:36.057806 3914 raft_consensus.cc:686] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:36.057926 3914 consensus_queue.cc:237] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:36.059290 2473 catalog_manager.cc:5484] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 0 to 1, leader changed from <none> to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:36.126235 3940 raft_consensus.cc:482] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:36.126391 3940 raft_consensus.cc:504] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:36.126837 3940 leader_election.cc:288] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:36.127315 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:36.127333 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" is_pre_election: true
I20240502 07:59:36.127457 2746 raft_consensus.cc:2454] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 0.
I20240502 07:59:36.127492 3008 raft_consensus.cc:2454] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 0.
I20240502 07:59:36.127774 2815 leader_election.cc:302] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:36.127936 3940 raft_consensus.cc:2781] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:36.128000 3940 raft_consensus.cc:482] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:36.128043 3940 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.128829 3940 raft_consensus.cc:504] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:36.129173 3940 leader_election.cc:288] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:36.129531 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:36.129562 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:36.129640 2746 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.129653 3008 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.130483 2746 raft_consensus.cc:2454] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 1.
I20240502 07:59:36.130749 3008 raft_consensus.cc:2454] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 1.
I20240502 07:59:36.130867 2815 leader_election.cc:302] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:36.131114 3940 raft_consensus.cc:2781] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:36.131214 3940 raft_consensus.cc:686] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:36.131356 3940 consensus_queue.cc:237] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:36.133234 2473 catalog_manager.cc:5484] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 0 to 1, leader changed from <none> to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 1 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:36.194926 3741 raft_consensus.cc:482] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:36.195063 3741 raft_consensus.cc:504] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:36.195472 3741 leader_election.cc:288] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:36.195924 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:36.195968 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" is_pre_election: true
I20240502 07:59:36.196055 2746 raft_consensus.cc:2454] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 0.
I20240502 07:59:36.196105 3008 raft_consensus.cc:2454] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 0.
I20240502 07:59:36.196390 2553 leader_election.cc:302] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:36.196568 3741 raft_consensus.cc:2781] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:36.196631 3741 raft_consensus.cc:482] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:36.196659 3741 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.197512 3741 raft_consensus.cc:504] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:36.197829 3741 leader_election.cc:288] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:36.198124 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:36.198127 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:36.198216 3008 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.198251 2746 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.198915 3008 raft_consensus.cc:2454] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 1.
I20240502 07:59:36.198931 2746 raft_consensus.cc:2454] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 1.
I20240502 07:59:36.199209 2550 leader_election.cc:302] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:36.199358 3741 raft_consensus.cc:2781] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:36.199432 3741 raft_consensus.cc:686] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:36.199579 3741 consensus_queue.cc:237] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:36.201311 2473 catalog_manager.cc:5484] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 0 to 1, leader changed from <none> to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 1 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
W20240502 07:59:36.355077 4013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:36.355319 4013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:36.358495 4013 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_6'
I20240502 07:59:36.438736 2746 raft_consensus.cc:1261] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:36.439361 3741 consensus_queue.cc:1035] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:36.440315 3008 raft_consensus.cc:1261] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:36.440696 3940 consensus_queue.cc:1035] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:36.441917 2747 raft_consensus.cc:1261] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:36.442615 3010 raft_consensus.cc:1261] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:36.442836 2748 raft_consensus.cc:1261] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:36.444773 4009 consensus_queue.cc:1035] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:36.445259 3741 consensus_queue.cc:1035] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:36.445477 2616 raft_consensus.cc:1261] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:36.449770 4009 consensus_queue.cc:1035] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:36.450191 3940 consensus_queue.cc:1035] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:36.458352 3984 mvcc.cc:205] Tried to move back new op lower bound from 7023152236301996032 to 7023152234734317568. Current Snapshot: MvccSnapshot[applied={T|T < 7023152236301996032}]
INSERT report
rows total: 2000
time total: 105.094 ms
time per row: 0.0525468 ms
I20240502 07:59:36.510811 2466 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53062:
name: "rebalance_test_table_7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:36.516737 2597 tablet_service.cc:1467] Processing CreateTablet for tablet 1afc3e666fcb4b1296f0d553f2835e0d (DEFAULT_TABLE table=rebalance_test_table_7 [id=9ae2583c897f469d8a41b1854c59f17d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:36.516737 2858 tablet_service.cc:1467] Processing CreateTablet for tablet 1afc3e666fcb4b1296f0d553f2835e0d (DEFAULT_TABLE table=rebalance_test_table_7 [id=9ae2583c897f469d8a41b1854c59f17d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:36.516842 2728 tablet_service.cc:1467] Processing CreateTablet for tablet 1afc3e666fcb4b1296f0d553f2835e0d (DEFAULT_TABLE table=rebalance_test_table_7 [id=9ae2583c897f469d8a41b1854c59f17d]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:36.517062 2858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1afc3e666fcb4b1296f0d553f2835e0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.517062 2597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1afc3e666fcb4b1296f0d553f2835e0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.517127 2728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1afc3e666fcb4b1296f0d553f2835e0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.518812 3996 tablet_bootstrap.cc:492] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:36.519230 3997 tablet_bootstrap.cc:492] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:36.519269 3998 tablet_bootstrap.cc:492] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:36.519804 3996 tablet_bootstrap.cc:654] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.520164 3997 tablet_bootstrap.cc:654] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.520180 3998 tablet_bootstrap.cc:654] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.520658 2858 tablet_service.cc:1467] Processing CreateTablet for tablet 7491c293498f4f93a23c62862dce44e5 (DEFAULT_TABLE table=rebalance_test_table_7 [id=9ae2583c897f469d8a41b1854c59f17d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:36.520885 2858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7491c293498f4f93a23c62862dce44e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.520854 2859 tablet_service.cc:1467] Processing CreateTablet for tablet fe897285bc7f4fcd8ab041aa9874b2ce (DEFAULT_TABLE table=rebalance_test_table_7 [id=9ae2583c897f469d8a41b1854c59f17d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:36.521257 3997 tablet_bootstrap.cc:492] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:36.521564 2859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe897285bc7f4fcd8ab041aa9874b2ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.521667 3997 ts_tablet_manager.cc:1367] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:36.522120 2597 tablet_service.cc:1467] Processing CreateTablet for tablet 7491c293498f4f93a23c62862dce44e5 (DEFAULT_TABLE table=rebalance_test_table_7 [id=9ae2583c897f469d8a41b1854c59f17d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:36.522320 3996 tablet_bootstrap.cc:492] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:36.522399 3996 ts_tablet_manager.cc:1367] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20240502 07:59:36.522562 2728 tablet_service.cc:1467] Processing CreateTablet for tablet fe897285bc7f4fcd8ab041aa9874b2ce (DEFAULT_TABLE table=rebalance_test_table_7 [id=9ae2583c897f469d8a41b1854c59f17d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:36.522655 2597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7491c293498f4f93a23c62862dce44e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.522784 2728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe897285bc7f4fcd8ab041aa9874b2ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.522828 3996 raft_consensus.cc:348] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:36.522929 3996 raft_consensus.cc:374] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.522961 3996 raft_consensus.cc:729] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.523092 3996 consensus_queue.cc:260] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:36.523353 3996 ts_tablet_manager.cc:1398] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:36.524119 2989 tablet_service.cc:1467] Processing CreateTablet for tablet 7491c293498f4f93a23c62862dce44e5 (DEFAULT_TABLE table=rebalance_test_table_7 [id=9ae2583c897f469d8a41b1854c59f17d]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:36.524384 2989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7491c293498f4f93a23c62862dce44e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.522208 3997 raft_consensus.cc:348] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:36.524915 3997 raft_consensus.cc:374] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.524966 3997 raft_consensus.cc:729] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.525097 3997 consensus_queue.cc:260] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:36.525408 3997 ts_tablet_manager.cc:1398] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:36.526340 3996 tablet_bootstrap.cc:492] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:36.526463 3997 tablet_bootstrap.cc:492] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:36.526652 2990 tablet_service.cc:1467] Processing CreateTablet for tablet fe897285bc7f4fcd8ab041aa9874b2ce (DEFAULT_TABLE table=rebalance_test_table_7 [id=9ae2583c897f469d8a41b1854c59f17d]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:36.526870 2990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe897285bc7f4fcd8ab041aa9874b2ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:36.527231 3996 tablet_bootstrap.cc:654] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.527313 3997 tablet_bootstrap.cc:654] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.528520 3997 tablet_bootstrap.cc:492] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:36.528558 3998 tablet_bootstrap.cc:492] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:36.528599 3997 ts_tablet_manager.cc:1367] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:36.528633 3998 ts_tablet_manager.cc:1367] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20240502 07:59:36.529013 3998 raft_consensus.cc:348] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:36.529026 3997 raft_consensus.cc:348] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:36.529124 3998 raft_consensus.cc:374] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.529132 3997 raft_consensus.cc:374] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.529148 4003 tablet_bootstrap.cc:492] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:36.529161 3998 raft_consensus.cc:729] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.529165 3997 raft_consensus.cc:729] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.529294 3998 consensus_queue.cc:260] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:36.529309 3997 consensus_queue.cc:260] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:36.529603 3998 ts_tablet_manager.cc:1398] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:36.529654 3997 ts_tablet_manager.cc:1398] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:36.529763 3998 tablet_bootstrap.cc:492] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:36.530061 4003 tablet_bootstrap.cc:654] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.530485 3998 tablet_bootstrap.cc:654] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.532528 4003 tablet_bootstrap.cc:492] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:36.532629 4003 ts_tablet_manager.cc:1367] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:36.532747 3996 tablet_bootstrap.cc:492] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:36.532827 3996 ts_tablet_manager.cc:1367] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20240502 07:59:36.533023 4003 raft_consensus.cc:348] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:36.533131 4003 raft_consensus.cc:374] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.533166 4003 raft_consensus.cc:729] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.533349 3996 raft_consensus.cc:348] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:36.533465 3996 raft_consensus.cc:374] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.533504 3996 raft_consensus.cc:729] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.533618 3996 consensus_queue.cc:260] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:36.533277 4003 consensus_queue.cc:260] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:36.533776 3998 tablet_bootstrap.cc:492] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:36.533849 3998 ts_tablet_manager.cc:1367] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:36.533900 3996 ts_tablet_manager.cc:1398] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:36.533906 4003 ts_tablet_manager.cc:1398] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:36.534071 4003 tablet_bootstrap.cc:492] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:36.534268 3998 raft_consensus.cc:348] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:36.534363 3998 raft_consensus.cc:374] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.534392 3998 raft_consensus.cc:729] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.534525 3998 consensus_queue.cc:260] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:36.534780 3998 ts_tablet_manager.cc:1398] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:36.534960 3998 tablet_bootstrap.cc:492] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:36.536546 3998 tablet_bootstrap.cc:654] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.537448 4003 tablet_bootstrap.cc:654] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:36.537902 3998 tablet_bootstrap.cc:492] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:36.538046 3998 ts_tablet_manager.cc:1367] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:36.538477 3998 raft_consensus.cc:348] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:36.538580 3998 raft_consensus.cc:374] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.538615 3998 raft_consensus.cc:729] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.538777 4003 tablet_bootstrap.cc:492] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:36.538751 3998 consensus_queue.cc:260] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:36.538854 4003 ts_tablet_manager.cc:1367] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20240502 07:59:36.539029 3998 ts_tablet_manager.cc:1398] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:36.539669 4003 raft_consensus.cc:348] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:36.539781 4003 raft_consensus.cc:374] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:36.539813 4003 raft_consensus.cc:729] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:36.539971 4003 consensus_queue.cc:260] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:36.540825 4003 ts_tablet_manager.cc:1398] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:36.562023 3843 raft_consensus.cc:482] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:36.562147 3843 raft_consensus.cc:504] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:36.562558 3843 leader_election.cc:288] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:36.562901 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" is_pre_election: true
I20240502 07:59:36.562902 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:36.563027 3010 raft_consensus.cc:2454] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 0.
I20240502 07:59:36.563056 2747 raft_consensus.cc:2454] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 0.
I20240502 07:59:36.563352 2815 leader_election.cc:302] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:36.563536 3843 raft_consensus.cc:2781] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:36.563597 3843 raft_consensus.cc:482] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:36.563642 3843 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.564646 3843 raft_consensus.cc:504] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:36.564977 3843 leader_election.cc:288] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:36.565291 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:36.565356 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:36.565447 3010 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.565436 2747 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.566378 2747 raft_consensus.cc:2454] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 1.
I20240502 07:59:36.566378 3010 raft_consensus.cc:2454] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 1.
I20240502 07:59:36.566650 2815 leader_election.cc:302] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:36.566901 3843 raft_consensus.cc:2781] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:36.566978 3843 raft_consensus.cc:686] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:36.567116 3843 consensus_queue.cc:237] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:36.568682 2466 catalog_manager.cc:5484] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 0 to 1, leader changed from <none> to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 1 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:36.633306 3843 raft_consensus.cc:482] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:36.633455 3843 raft_consensus.cc:504] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:36.633872 3843 leader_election.cc:288] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:36.634258 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" is_pre_election: true
I20240502 07:59:36.634264 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b" is_pre_election: true
I20240502 07:59:36.634405 2616 raft_consensus.cc:2454] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 0.
I20240502 07:59:36.634404 3010 raft_consensus.cc:2454] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 0.
I20240502 07:59:36.634677 2812 leader_election.cc:302] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:36.634835 3843 raft_consensus.cc:2781] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:36.634892 3843 raft_consensus.cc:482] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:36.634936 3843 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.635777 3843 raft_consensus.cc:504] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:36.636075 3843 leader_election.cc:288] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:36.636411 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:36.636420 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:36.636519 2616 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.636555 3010 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.637351 2616 raft_consensus.cc:2454] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 1.
I20240502 07:59:36.637395 3010 raft_consensus.cc:2454] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 1.
I20240502 07:59:36.637670 2812 leader_election.cc:302] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:36.637827 3843 raft_consensus.cc:2781] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:36.637907 3843 raft_consensus.cc:686] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:36.638053 3843 consensus_queue.cc:237] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:36.639853 2466 catalog_manager.cc:5484] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 0 to 1, leader changed from <none> to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 1 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:36.954453 3843 raft_consensus.cc:482] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:36.954658 3843 raft_consensus.cc:504] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:36.955282 3843 leader_election.cc:288] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:36.955924 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b" is_pre_election: true
I20240502 07:59:36.956080 2616 raft_consensus.cc:2454] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 0.
I20240502 07:59:36.956523 2813 leader_election.cc:302] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:36.956740 3843 raft_consensus.cc:2781] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:36.956799 3843 raft_consensus.cc:482] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:36.956846 3843 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.957975 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:36.957974 3843 raft_consensus.cc:504] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:36.958115 2747 raft_consensus.cc:2454] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 0.
I20240502 07:59:36.958593 3843 leader_election.cc:288] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:36.958601 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:36.958710 2616 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.959420 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:36.959523 2747 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:36.959723 2616 raft_consensus.cc:2454] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 1.
I20240502 07:59:36.959993 2813 leader_election.cc:302] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:36.960139 3843 raft_consensus.cc:2781] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:36.960223 3843 raft_consensus.cc:686] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:36.960371 3843 consensus_queue.cc:237] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:36.960418 2747 raft_consensus.cc:2454] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 1.
I20240502 07:59:36.961990 2466 catalog_manager.cc:5484] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 0 to 1, leader changed from <none> to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 1 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:36.981827 3843 consensus_queue.cc:1035] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:36.984953 3843 consensus_queue.cc:1035] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:37.067692 3843 consensus_queue.cc:1035] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:37.071056 3940 consensus_queue.cc:1035] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240502 07:59:37.084975 4065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:37.085204 4065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:37.088346 4065 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_7'
I20240502 07:59:37.168228 2617 raft_consensus.cc:1261] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:37.168691 3940 consensus_queue.cc:1035] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:37.170212 2746 raft_consensus.cc:1261] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:37.171334 3843 consensus_queue.cc:1035] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 94.3773 ms
time per row: 0.0471886 ms
I20240502 07:59:37.226655 2466 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53062:
name: "rebalance_test_table_8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:37.231943 2728 tablet_service.cc:1467] Processing CreateTablet for tablet 001fcdef92e8443baf7ac443a1b4c57a (DEFAULT_TABLE table=rebalance_test_table_8 [id=f9fed45f50a54e21bd1dda67a4bde334]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:37.232038 2858 tablet_service.cc:1467] Processing CreateTablet for tablet 001fcdef92e8443baf7ac443a1b4c57a (DEFAULT_TABLE table=rebalance_test_table_8 [id=f9fed45f50a54e21bd1dda67a4bde334]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:37.232296 2728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 001fcdef92e8443baf7ac443a1b4c57a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:37.232347 2858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 001fcdef92e8443baf7ac443a1b4c57a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:37.232461 2990 tablet_service.cc:1467] Processing CreateTablet for tablet f116334237934021a25a6cacae430034 (DEFAULT_TABLE table=rebalance_test_table_8 [id=f9fed45f50a54e21bd1dda67a4bde334]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:37.232710 2990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f116334237934021a25a6cacae430034. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:37.232784 2989 tablet_service.cc:1467] Processing CreateTablet for tablet 18912e0059054584a990be7ccb4a5771 (DEFAULT_TABLE table=rebalance_test_table_8 [id=f9fed45f50a54e21bd1dda67a4bde334]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:37.233016 2989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18912e0059054584a990be7ccb4a5771. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:37.235051 4126 tablet_bootstrap.cc:492] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:37.235355 2858 tablet_service.cc:1467] Processing CreateTablet for tablet 18912e0059054584a990be7ccb4a5771 (DEFAULT_TABLE table=rebalance_test_table_8 [id=f9fed45f50a54e21bd1dda67a4bde334]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:37.235483 2597 tablet_service.cc:1467] Processing CreateTablet for tablet 001fcdef92e8443baf7ac443a1b4c57a (DEFAULT_TABLE table=rebalance_test_table_8 [id=f9fed45f50a54e21bd1dda67a4bde334]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:37.235594 2858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18912e0059054584a990be7ccb4a5771. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:37.235682 2595 tablet_service.cc:1467] Processing CreateTablet for tablet 18912e0059054584a990be7ccb4a5771 (DEFAULT_TABLE table=rebalance_test_table_8 [id=f9fed45f50a54e21bd1dda67a4bde334]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:37.235756 2597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 001fcdef92e8443baf7ac443a1b4c57a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:37.235909 2595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18912e0059054584a990be7ccb4a5771. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:37.236094 4126 tablet_bootstrap.cc:654] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:37.236140 2596 tablet_service.cc:1467] Processing CreateTablet for tablet f116334237934021a25a6cacae430034 (DEFAULT_TABLE table=rebalance_test_table_8 [id=f9fed45f50a54e21bd1dda67a4bde334]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:37.236356 2596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f116334237934021a25a6cacae430034. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:37.236718 4125 tablet_bootstrap.cc:492] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:37.237613 4125 tablet_bootstrap.cc:654] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:37.238708 2728 tablet_service.cc:1467] Processing CreateTablet for tablet f116334237934021a25a6cacae430034 (DEFAULT_TABLE table=rebalance_test_table_8 [id=f9fed45f50a54e21bd1dda67a4bde334]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:37.238826 4129 tablet_bootstrap.cc:492] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:37.238884 4126 tablet_bootstrap.cc:492] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:37.238940 2728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f116334237934021a25a6cacae430034. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:37.238962 4126 ts_tablet_manager.cc:1367] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20240502 07:59:37.239414 4126 raft_consensus.cc:348] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.239523 4126 raft_consensus.cc:374] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:37.239553 4126 raft_consensus.cc:729] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:37.239744 4129 tablet_bootstrap.cc:654] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:37.239689 4126 consensus_queue.cc:260] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.240700 4125 tablet_bootstrap.cc:492] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:37.240826 4125 ts_tablet_manager.cc:1367] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20240502 07:59:37.240866 4126 ts_tablet_manager.cc:1398] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:37.241039 4126 tablet_bootstrap.cc:492] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:37.241096 4129 tablet_bootstrap.cc:492] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:37.241189 4129 ts_tablet_manager.cc:1367] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:37.241515 4125 raft_consensus.cc:348] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:37.241681 4125 raft_consensus.cc:374] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:37.241703 4129 raft_consensus.cc:348] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.241763 4125 raft_consensus.cc:729] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:37.241823 4129 raft_consensus.cc:374] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:37.241861 4129 raft_consensus.cc:729] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:37.241986 4129 consensus_queue.cc:260] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.242045 4125 consensus_queue.cc:260] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:37.242259 4129 ts_tablet_manager.cc:1398] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:37.242424 4129 tablet_bootstrap.cc:492] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:37.242522 4125 ts_tablet_manager.cc:1398] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:37.242729 4125 tablet_bootstrap.cc:492] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:37.243276 4129 tablet_bootstrap.cc:654] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:37.243409 4126 tablet_bootstrap.cc:654] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:37.243716 4125 tablet_bootstrap.cc:654] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:37.244597 4129 tablet_bootstrap.cc:492] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:37.244688 4129 ts_tablet_manager.cc:1367] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:37.245132 4129 raft_consensus.cc:348] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:37.245234 4129 raft_consensus.cc:374] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:37.245273 4129 raft_consensus.cc:729] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:37.245429 4129 consensus_queue.cc:260] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:37.245772 4129 ts_tablet_manager.cc:1398] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:37.245806 4125 tablet_bootstrap.cc:492] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:37.245898 4125 ts_tablet_manager.cc:1367] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:37.245924 4129 tablet_bootstrap.cc:492] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:37.245909 4124 tablet_bootstrap.cc:492] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:37.246480 4125 raft_consensus.cc:348] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:37.246613 4125 raft_consensus.cc:374] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:37.246659 4125 raft_consensus.cc:729] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:37.246778 4129 tablet_bootstrap.cc:654] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:37.246779 4125 consensus_queue.cc:260] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:37.247066 4125 ts_tablet_manager.cc:1398] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:37.247222 4124 tablet_bootstrap.cc:654] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:37.248001 4129 tablet_bootstrap.cc:492] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:37.248271 4124 tablet_bootstrap.cc:492] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:37.248355 4124 ts_tablet_manager.cc:1367] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:37.248526 4126 tablet_bootstrap.cc:492] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:37.248603 4126 ts_tablet_manager.cc:1367] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20240502 07:59:37.248783 4124 raft_consensus.cc:348] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:37.248886 4124 raft_consensus.cc:374] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:37.248925 4124 raft_consensus.cc:729] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:37.248852 4129 ts_tablet_manager.cc:1367] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 07:59:37.249022 4126 raft_consensus.cc:348] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:37.249040 4124 consensus_queue.cc:260] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:37.249126 4126 raft_consensus.cc:374] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:37.249167 4126 raft_consensus.cc:729] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:37.249296 4126 consensus_queue.cc:260] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:37.249471 4124 ts_tablet_manager.cc:1398] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:37.249609 4124 tablet_bootstrap.cc:492] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:37.250064 4126 ts_tablet_manager.cc:1398] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:37.250452 4124 tablet_bootstrap.cc:654] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:37.250671 4129 raft_consensus.cc:348] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:37.250830 4129 raft_consensus.cc:374] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:37.250909 4129 raft_consensus.cc:729] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:37.251513 4129 consensus_queue.cc:260] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:37.251998 4129 ts_tablet_manager.cc:1398] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:37.252316 4124 tablet_bootstrap.cc:492] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:37.252390 4124 ts_tablet_manager.cc:1367] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20240502 07:59:37.252790 4124 raft_consensus.cc:348] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.252878 4124 raft_consensus.cc:374] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:37.252911 4124 raft_consensus.cc:729] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:37.253021 4124 consensus_queue.cc:260] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.253252 4124 ts_tablet_manager.cc:1398] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:37.279758 4044 raft_consensus.cc:482] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:37.279883 4044 raft_consensus.cc:504] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.280326 4044 leader_election.cc:288] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:37.280706 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:37.280709 2617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b" is_pre_election: true
I20240502 07:59:37.280841 2746 raft_consensus.cc:2454] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 0.
I20240502 07:59:37.280874 2617 raft_consensus.cc:2454] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 0.
I20240502 07:59:37.281170 2946 leader_election.cc:302] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:37.281379 4044 raft_consensus.cc:2781] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:37.281440 4044 raft_consensus.cc:482] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:37.281484 4044 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:37.282316 4044 raft_consensus.cc:504] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.282629 4044 leader_election.cc:288] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:37.282958 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:37.282958 2617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:37.283073 2617 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:37.283073 2746 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:37.284009 2617 raft_consensus.cc:2454] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 1.
I20240502 07:59:37.284009 2746 raft_consensus.cc:2454] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 1.
I20240502 07:59:37.284284 2944 leader_election.cc:302] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:37.284444 4044 raft_consensus.cc:2781] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:37.284682 4044 raft_consensus.cc:686] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:37.284852 4044 consensus_queue.cc:237] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.286749 2466 catalog_manager.cc:5484] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 0 to 1, leader changed from <none> to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:37.307669 4044 raft_consensus.cc:482] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:37.307797 4044 raft_consensus.cc:504] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:37.308262 4044 leader_election.cc:288] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:37.308568 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681" is_pre_election: true
I20240502 07:59:37.308717 2879 raft_consensus.cc:2454] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 0.
I20240502 07:59:37.308714 2617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b" is_pre_election: true
I20240502 07:59:37.308869 2617 raft_consensus.cc:2454] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 0.
I20240502 07:59:37.309036 2944 leader_election.cc:302] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:37.309195 4044 raft_consensus.cc:2781] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:37.309258 4044 raft_consensus.cc:482] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:37.309298 4044 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:37.310132 4044 raft_consensus.cc:504] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:37.310448 4044 leader_election.cc:288] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:37.310745 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:37.310839 2879 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:37.310891 2617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:37.311020 2617 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:37.311733 2879 raft_consensus.cc:2454] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 1.
I20240502 07:59:37.312155 2617 raft_consensus.cc:2454] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 1.
I20240502 07:59:37.312155 2944 leader_election.cc:302] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:37.312361 4044 raft_consensus.cc:2781] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:37.312426 4044 raft_consensus.cc:686] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:37.312552 4044 consensus_queue.cc:237] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:37.312956 3741 raft_consensus.cc:482] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:37.313036 3741 raft_consensus.cc:504] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:37.313788 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:37.313906 2746 raft_consensus.cc:2454] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 0.
I20240502 07:59:37.314172 2553 leader_election.cc:302] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:37.314334 4074 raft_consensus.cc:2781] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:37.314374 3741 leader_election.cc:288] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:37.314404 4074 raft_consensus.cc:482] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:37.314486 4074 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:37.314488 2466 catalog_manager.cc:5484] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 0 to 1, leader changed from <none> to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:37.314723 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681" is_pre_election: true
I20240502 07:59:37.314834 2879 raft_consensus.cc:2454] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 0.
I20240502 07:59:37.315385 4074 raft_consensus.cc:504] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:37.315712 4074 leader_election.cc:288] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:37.315955 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:37.316057 2746 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:37.316247 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:37.316335 2879 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:37.316798 2746 raft_consensus.cc:2454] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 1.
I20240502 07:59:37.317028 2879 raft_consensus.cc:2454] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 1.
I20240502 07:59:37.317123 2553 leader_election.cc:302] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:37.317341 4074 raft_consensus.cc:2781] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:37.317446 4074 raft_consensus.cc:686] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:37.317576 4074 consensus_queue.cc:237] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:37.319100 2475 catalog_manager.cc:5484] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 0 to 1, leader changed from <none> to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 1 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
W20240502 07:59:37.449558 4141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:37.449792 4141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:37.452991 4141 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_8'
I20240502 07:59:37.533412 2746 raft_consensus.cc:1261] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:37.533874 4074 consensus_queue.cc:1035] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:37.534355 2617 raft_consensus.cc:1261] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:37.534385 2879 raft_consensus.cc:1261] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:37.534713 2878 raft_consensus.cc:1261] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:37.535039 4172 consensus_queue.cc:1035] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:37.535871 2747 raft_consensus.cc:1261] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:37.537070 4074 consensus_queue.cc:1035] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:37.538589 4137 consensus_queue.cc:1035] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:37.539002 4137 consensus_queue.cc:1035] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:37.542436 2617 raft_consensus.cc:1261] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:37.553660 4172 consensus_queue.cc:1035] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 113.152 ms
time per row: 0.0565759 ms
I20240502 07:59:37.608228 2466 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53062:
name: "rebalance_test_table_9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
I20240502 07:59:37.613619 2728 tablet_service.cc:1467] Processing CreateTablet for tablet c67533803b9c496295f4f57e77a51374 (DEFAULT_TABLE table=rebalance_test_table_9 [id=015591fd938646a1ba825dc165b529b5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:37.613950 2728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c67533803b9c496295f4f57e77a51374. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:37.613920 2990 tablet_service.cc:1467] Processing CreateTablet for tablet 512e1322925c415f8891dfe6621160e1 (DEFAULT_TABLE table=rebalance_test_table_9 [id=015591fd938646a1ba825dc165b529b5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:37.614192 2990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 512e1322925c415f8891dfe6621160e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:37.614187 2726 tablet_service.cc:1467] Processing CreateTablet for tablet 512e1322925c415f8891dfe6621160e1 (DEFAULT_TABLE table=rebalance_test_table_9 [id=015591fd938646a1ba825dc165b529b5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:37.614404 2726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 512e1322925c415f8891dfe6621160e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:37.615571 4124 tablet_bootstrap.cc:492] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:37.615759 2989 tablet_service.cc:1467] Processing CreateTablet for tablet c67533803b9c496295f4f57e77a51374 (DEFAULT_TABLE table=rebalance_test_table_9 [id=015591fd938646a1ba825dc165b529b5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:37.615971 2989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c67533803b9c496295f4f57e77a51374. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:37.616298 4126 tablet_bootstrap.cc:492] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:37.616381 4124 tablet_bootstrap.cc:654] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:37.616973 2858 tablet_service.cc:1467] Processing CreateTablet for tablet c67533803b9c496295f4f57e77a51374 (DEFAULT_TABLE table=rebalance_test_table_9 [id=015591fd938646a1ba825dc165b529b5]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:37.617100 4126 tablet_bootstrap.cc:654] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:37.617249 2858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c67533803b9c496295f4f57e77a51374. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:37.617826 2857 tablet_service.cc:1467] Processing CreateTablet for tablet 37b0b5493f8748338d5c373eaee489d4 (DEFAULT_TABLE table=rebalance_test_table_9 [id=015591fd938646a1ba825dc165b529b5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:37.617914 2727 tablet_service.cc:1467] Processing CreateTablet for tablet 37b0b5493f8748338d5c373eaee489d4 (DEFAULT_TABLE table=rebalance_test_table_9 [id=015591fd938646a1ba825dc165b529b5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:37.618042 2857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37b0b5493f8748338d5c373eaee489d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:37.618160 2727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37b0b5493f8748338d5c373eaee489d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:37.618384 4126 tablet_bootstrap.cc:492] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:37.618458 4126 ts_tablet_manager.cc:1367] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20240502 07:59:37.618850 4126 raft_consensus.cc:348] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.618947 4126 raft_consensus.cc:374] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:37.618979 4126 raft_consensus.cc:729] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:37.619107 4126 consensus_queue.cc:260] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.619344 4125 tablet_bootstrap.cc:492] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:37.619398 4126 ts_tablet_manager.cc:1398] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:37.619549 4126 tablet_bootstrap.cc:492] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef: Bootstrap starting.
I20240502 07:59:37.620149 2859 tablet_service.cc:1467] Processing CreateTablet for tablet 512e1322925c415f8891dfe6621160e1 (DEFAULT_TABLE table=rebalance_test_table_9 [id=015591fd938646a1ba825dc165b529b5]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:37.620322 4125 tablet_bootstrap.cc:654] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:37.620378 2859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 512e1322925c415f8891dfe6621160e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:37.620393 4126 tablet_bootstrap.cc:654] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:37.621523 4125 tablet_bootstrap.cc:492] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:37.621610 4125 ts_tablet_manager.cc:1367] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:37.622071 4125 raft_consensus.cc:348] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:37.622185 4125 raft_consensus.cc:374] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:37.622236 4125 raft_consensus.cc:729] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:37.622380 4125 consensus_queue.cc:260] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:37.622504 4126 tablet_bootstrap.cc:492] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef: No bootstrap required, opened a new log
I20240502 07:59:37.622587 4126 ts_tablet_manager.cc:1367] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20240502 07:59:37.622632 4125 ts_tablet_manager.cc:1398] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:37.622730 4124 tablet_bootstrap.cc:492] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:37.622793 4125 tablet_bootstrap.cc:492] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:37.622804 4124 ts_tablet_manager.cc:1367] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20240502 07:59:37.623009 4126 raft_consensus.cc:348] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:37.623091 4126 raft_consensus.cc:374] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:37.623121 4126 raft_consensus.cc:729] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Initialized, Role: FOLLOWER
I20240502 07:59:37.623201 4124 raft_consensus.cc:348] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:37.623245 4126 consensus_queue.cc:260] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:37.623301 4124 raft_consensus.cc:374] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:37.623334 4124 raft_consensus.cc:729] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:37.623451 4124 consensus_queue.cc:260] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:37.623600 4125 tablet_bootstrap.cc:654] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:37.623724 4124 ts_tablet_manager.cc:1398] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:37.623920 4124 tablet_bootstrap.cc:492] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:37.623929 4126 ts_tablet_manager.cc:1398] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:37.624385 2597 tablet_service.cc:1467] Processing CreateTablet for tablet 37b0b5493f8748338d5c373eaee489d4 (DEFAULT_TABLE table=rebalance_test_table_9 [id=015591fd938646a1ba825dc165b529b5]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20240502 07:59:37.624636 2597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37b0b5493f8748338d5c373eaee489d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20240502 07:59:37.624640 4125 tablet_bootstrap.cc:492] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:37.624722 4124 tablet_bootstrap.cc:654] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:37.624716 4125 ts_tablet_manager.cc:1367] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20240502 07:59:37.625140 4125 raft_consensus.cc:348] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.625247 4125 raft_consensus.cc:374] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:37.625293 4125 raft_consensus.cc:729] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:37.625473 4125 consensus_queue.cc:260] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.625723 4125 ts_tablet_manager.cc:1398] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20240502 07:59:37.626035 4125 tablet_bootstrap.cc:492] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681: Bootstrap starting.
I20240502 07:59:37.626883 4129 tablet_bootstrap.cc:492] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b: Bootstrap starting.
I20240502 07:59:37.626907 4125 tablet_bootstrap.cc:654] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:37.626960 4124 tablet_bootstrap.cc:492] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:37.627043 4124 ts_tablet_manager.cc:1367] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20240502 07:59:37.627508 4124 raft_consensus.cc:348] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.627610 4124 raft_consensus.cc:374] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:37.627658 4124 raft_consensus.cc:729] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:37.627818 4129 tablet_bootstrap.cc:654] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:37.627792 4124 consensus_queue.cc:260] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.629801 4129 tablet_bootstrap.cc:492] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b: No bootstrap required, opened a new log
I20240502 07:59:37.629896 4129 ts_tablet_manager.cc:1367] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20240502 07:59:37.629910 4125 tablet_bootstrap.cc:492] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681: No bootstrap required, opened a new log
I20240502 07:59:37.629988 4125 ts_tablet_manager.cc:1367] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20240502 07:59:37.630436 4125 raft_consensus.cc:348] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.630385 4129 raft_consensus.cc:348] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.630539 4125 raft_consensus.cc:374] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:37.630540 4129 raft_consensus.cc:374] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:37.630581 4129 raft_consensus.cc:729] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Initialized, Role: FOLLOWER
I20240502 07:59:37.630579 4125 raft_consensus.cc:729] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Initialized, Role: FOLLOWER
I20240502 07:59:37.630698 4125 consensus_queue.cc:260] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.630710 4129 consensus_queue.cc:260] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.630981 4129 ts_tablet_manager.cc:1398] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20240502 07:59:37.630988 4125 ts_tablet_manager.cc:1398] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:37.633437 4124 ts_tablet_manager.cc:1398] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
I20240502 07:59:37.634439 4124 tablet_bootstrap.cc:492] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1: Bootstrap starting.
I20240502 07:59:37.636189 4124 tablet_bootstrap.cc:654] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1: Neither blocks nor log segments found. Creating new log.
I20240502 07:59:37.637486 4124 tablet_bootstrap.cc:492] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1: No bootstrap required, opened a new log
I20240502 07:59:37.637568 4124 ts_tablet_manager.cc:1367] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20240502 07:59:37.637984 4124 raft_consensus.cc:348] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.638113 4124 raft_consensus.cc:374] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240502 07:59:37.638154 4124 raft_consensus.cc:729] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Initialized, Role: FOLLOWER
I20240502 07:59:37.638273 4124 consensus_queue.cc:260] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.638556 4124 ts_tablet_manager.cc:1398] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20240502 07:59:37.639837 4171 raft_consensus.cc:482] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:37.639935 4171 raft_consensus.cc:504] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:37.640307 4171 leader_election.cc:288] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:37.640589 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:37.640700 2746 raft_consensus.cc:2454] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 0.
I20240502 07:59:37.640980 2946 leader_election.cc:302] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:37.641026 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681" is_pre_election: true
I20240502 07:59:37.641139 2879 raft_consensus.cc:2454] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 0.
I20240502 07:59:37.641144 4171 raft_consensus.cc:2781] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:37.641196 4171 raft_consensus.cc:482] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:37.641237 4171 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:37.642087 4171 raft_consensus.cc:504] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:37.642380 4171 leader_election.cc:288] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:37.642652 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:37.642699 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:37.642752 2746 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:37.642776 2879 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:37.643461 2879 raft_consensus.cc:2454] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 1.
I20240502 07:59:37.643659 2746 raft_consensus.cc:2454] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 1.
I20240502 07:59:37.643852 2944 leader_election.cc:302] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:37.644014 4171 raft_consensus.cc:2781] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:37.644101 4171 raft_consensus.cc:686] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:37.644224 4171 consensus_queue.cc:237] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:37.646011 2473 catalog_manager.cc:5484] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 0 to 1, leader changed from <none> to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 1 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:37.724557 3741 raft_consensus.cc:482] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:37.724696 3741 raft_consensus.cc:504] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.725114 3741 leader_election.cc:288] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:37.725515 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681" is_pre_election: true
I20240502 07:59:37.725517 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1" is_pre_election: true
I20240502 07:59:37.725647 2746 raft_consensus.cc:2454] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 0.
I20240502 07:59:37.725647 2879 raft_consensus.cc:2454] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 0.
I20240502 07:59:37.725941 2553 leader_election.cc:302] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:37.726166 3741 raft_consensus.cc:2781] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:37.726244 3741 raft_consensus.cc:482] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:37.726285 3741 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:37.727063 3741 raft_consensus.cc:504] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.727377 3741 leader_election.cc:288] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:37.727670 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:37.727695 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:37.727771 2879 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:37.727783 2746 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:37.728647 2746 raft_consensus.cc:2454] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 1.
I20240502 07:59:37.728684 2879 raft_consensus.cc:2454] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 1.
I20240502 07:59:37.728946 2553 leader_election.cc:302] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:37.729132 3741 raft_consensus.cc:2781] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:37.729252 3741 raft_consensus.cc:686] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:37.729411 3741 consensus_queue.cc:237] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.731113 2473 catalog_manager.cc:5484] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 0 to 1, leader changed from <none> to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 1 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:37.981884 4069 raft_consensus.cc:482] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240502 07:59:37.982023 4069 raft_consensus.cc:504] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.982555 4069 leader_election.cc:288] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:37.982967 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" is_pre_election: true
I20240502 07:59:37.983129 3010 raft_consensus.cc:2454] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 0.
I20240502 07:59:37.983472 2681 leader_election.cc:302] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:37.983649 4069 raft_consensus.cc:2781] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240502 07:59:37.983716 4069 raft_consensus.cc:482] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240502 07:59:37.983765 4069 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:37.984848 4069 raft_consensus.cc:504] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.985172 4069 leader_election.cc:288] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 1 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:37.985553 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:37.985661 3010 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:37.986434 3010 raft_consensus.cc:2454] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 1.
I20240502 07:59:37.986662 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:37.986704 2681 leader_election.cc:302] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:37.986682 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "606c439025f24e53a2feebeaf68c1681" is_pre_election: true
I20240502 07:59:37.986815 2878 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 0 FOLLOWER]: Advancing to term 1
I20240502 07:59:37.986874 4069 raft_consensus.cc:2781] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Leader election won for term 1
I20240502 07:59:37.987018 4069 raft_consensus.cc:686] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 1 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:37.987172 4069 consensus_queue.cc:237] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:37.987865 2878 raft_consensus.cc:2454] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 1.
I20240502 07:59:37.989434 2473 catalog_manager.cc:5484] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 1 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
W20240502 07:59:38.103188 4195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:38.103436 4195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:38.106645 4195 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
Using table 'rebalance_test_table_9'
I20240502 07:59:38.136885 4171 consensus_queue.cc:1035] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:38.141498 4171 consensus_queue.cc:1035] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:38.169656 3741 consensus_queue.cc:1035] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:38.172912 4074 consensus_queue.cc:1035] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:38.193223 2878 raft_consensus.cc:1261] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:38.193625 3010 raft_consensus.cc:1261] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240502 07:59:38.194551 4193 consensus_queue.cc:1035] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240502 07:59:38.194687 4069 consensus_queue.cc:1035] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
INSERT report
rows total: 2000
time total: 106.103 ms
time per row: 0.0530515 ms
I20240502 07:59:38.257890 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.133:44673
--local_ip_for_outbound_sockets=127.15.129.133
--tserver_master_addrs=127.15.129.190:34757
--webserver_port=44433
--webserver_interface=127.15.129.133
--builtin_ntp_servers=127.15.129.148:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
W20240502 07:59:38.373481 4258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:38.373703 4258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:38.373750 4258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:38.376751 4258 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:59:38.376811 4258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:38.376876 4258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.133
I20240502 07:59:38.380477 4258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.133:44673
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.15.129.133
--webserver_port=44433
--tserver_master_addrs=127.15.129.190:34757
--never_fsync=true
--heap_profile_path=/tmp/kudu.4258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.133
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:38.381418 4258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:38.382515 4258 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:38.391299 4263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:38.391360 4264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:38.392071 4266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:38.392992 4258 server_base.cc:1034] running on GCE node
I20240502 07:59:38.393468 4258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:38.394073 4258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:38.395228 4258 hybrid_clock.cc:648] HybridClock initialized: now 1714636778395216 us; error 30 us; skew 500 ppm
I20240502 07:59:38.402678 4258 webserver.cc:460] Webserver started at http://127.15.129.133:44433/ using document root <none> and password file <none>
I20240502 07:59:38.403281 4258 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:38.403367 4258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:38.406984 4258 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:59:38.409090 4272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:38.410192 4258 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:38.410305 4258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "4507e2f07af94d7cabcbe75708efa8d6"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:38.410760 4258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:38.425958 4258 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:38.426611 4258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:38.426788 4258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:38.427325 4258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:38.428398 4258 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:38.428479 4258 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:38.428560 4258 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:38.428594 4258 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:38.438675 4258 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.133:44673
I20240502 07:59:38.438812 4385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.133:44673 every 8 connection(s)
I20240502 07:59:38.439888 4258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20240502 07:59:38.444087 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 4258
I20240502 07:59:38.444572 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.134:44735
--local_ip_for_outbound_sockets=127.15.129.134
--tserver_master_addrs=127.15.129.190:34757
--webserver_port=44541
--webserver_interface=127.15.129.134
--builtin_ntp_servers=127.15.129.148:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:59:38.451160 4386 heartbeater.cc:347] Connected to a master server at 127.15.129.190:34757
I20240502 07:59:38.451407 4386 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:38.451897 4386 heartbeater.cc:511] Master 127.15.129.190:34757 requested a full tablet report, sending...
I20240502 07:59:38.453109 2470 ts_manager.cc:194] Re-registered known tserver with Master: 4507e2f07af94d7cabcbe75708efa8d6 (127.15.129.133:44673)
I20240502 07:59:38.453964 2470 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.133:46917
W20240502 07:59:38.559543 4389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:38.559830 4389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:38.559923 4389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:38.563447 4389 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:59:38.563524 4389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:38.563630 4389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.134
I20240502 07:59:38.567449 4389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.134:44735
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.15.129.134
--webserver_port=44541
--tserver_master_addrs=127.15.129.190:34757
--never_fsync=true
--heap_profile_path=/tmp/kudu.4389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.134
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:38.568360 4389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:38.569486 4389 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:38.577301 4395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:38.577891 4394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:38.578452 4397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:38.579329 4389 server_base.cc:1034] running on GCE node
I20240502 07:59:38.579783 4389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:38.580340 4389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:38.581490 4389 hybrid_clock.cc:648] HybridClock initialized: now 1714636778581458 us; error 47 us; skew 500 ppm
I20240502 07:59:38.584102 4389 webserver.cc:460] Webserver started at http://127.15.129.134:44541/ using document root <none> and password file <none>
I20240502 07:59:38.584820 4389 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:38.584906 4389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:38.599088 4389 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.003s sys 0.000s
I20240502 07:59:38.602025 4403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:38.603871 4389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20240502 07:59:38.604022 4389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "9598319af8734926bdcfcd65c7dac9f1"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:38.604672 4389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:38.628170 4389 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:38.628834 4389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:38.629034 4389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:38.629647 4389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:38.630613 4389 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:38.630659 4389 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:38.630699 4389 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:38.630733 4389 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:38.641745 4389 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.134:44735
I20240502 07:59:38.641778 4516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.134:44735 every 8 connection(s)
I20240502 07:59:38.642802 4389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20240502 07:59:38.644819 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 4389
I20240502 07:59:38.645352 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.135:41793
--local_ip_for_outbound_sockets=127.15.129.135
--tserver_master_addrs=127.15.129.190:34757
--webserver_port=45967
--webserver_interface=127.15.129.135
--builtin_ntp_servers=127.15.129.148:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:59:38.653091 4517 heartbeater.cc:347] Connected to a master server at 127.15.129.190:34757
I20240502 07:59:38.653349 4517 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:38.653812 4517 heartbeater.cc:511] Master 127.15.129.190:34757 requested a full tablet report, sending...
I20240502 07:59:38.654881 2470 ts_manager.cc:194] Re-registered known tserver with Master: 9598319af8734926bdcfcd65c7dac9f1 (127.15.129.134:44735)
I20240502 07:59:38.655643 2470 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.134:34137
W20240502 07:59:38.766999 4520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:38.767235 4520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:38.767309 4520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:38.770799 4520 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:59:38.770877 4520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:38.770951 4520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.135
I20240502 07:59:38.774576 4520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.135:41793
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.15.129.135
--webserver_port=45967
--tserver_master_addrs=127.15.129.190:34757
--never_fsync=true
--heap_profile_path=/tmp/kudu.4520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.135
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:38.775614 4520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:38.776886 4520 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:38.784869 4527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:38.785526 4530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:38.786360 4520 server_base.cc:1034] running on GCE node
W20240502 07:59:38.786887 4528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:38.787264 4520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:38.787829 4520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:38.788969 4520 hybrid_clock.cc:648] HybridClock initialized: now 1714636778788931 us; error 53 us; skew 500 ppm
I20240502 07:59:38.790858 4520 webserver.cc:460] Webserver started at http://127.15.129.135:45967/ using document root <none> and password file <none>
I20240502 07:59:38.791443 4520 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:38.791517 4520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:38.794992 4520 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20240502 07:59:38.796805 4536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:38.797821 4520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:38.797924 4520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "ab75717712534dd7b35c2e432d65533e"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:38.798329 4520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:38.821874 4520 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:38.822494 4520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:38.822654 4520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:38.823189 4520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:38.824034 4520 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:38.824081 4520 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:38.824137 4520 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:38.824169 4520 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:38.832324 4520 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.135:41793
I20240502 07:59:38.832366 4649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.135:41793 every 8 connection(s)
I20240502 07:59:38.833346 4520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20240502 07:59:38.842236 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 4520
I20240502 07:59:38.842936 15878 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
/tmp/dist-test-taskUnCtd5/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/logs
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.129.136:41115
--local_ip_for_outbound_sockets=127.15.129.136
--tserver_master_addrs=127.15.129.190:34757
--webserver_port=42365
--webserver_interface=127.15.129.136
--builtin_ntp_servers=127.15.129.148:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false with env {}
I20240502 07:59:38.845597 4650 heartbeater.cc:347] Connected to a master server at 127.15.129.190:34757
I20240502 07:59:38.845834 4650 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:38.846295 4650 heartbeater.cc:511] Master 127.15.129.190:34757 requested a full tablet report, sending...
I20240502 07:59:38.847271 2470 ts_manager.cc:194] Re-registered known tserver with Master: ab75717712534dd7b35c2e432d65533e (127.15.129.135:41793)
I20240502 07:59:38.848119 2470 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.135:34423
W20240502 07:59:38.964726 4653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240502 07:59:38.964951 4653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240502 07:59:38.965003 4653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240502 07:59:38.968067 4653 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20240502 07:59:38.968152 4653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240502 07:59:38.968269 4653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.129.136
I20240502 07:59:38.972121 4653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.129.148:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data
--fs_wal_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.129.136:41115
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/info.pb
--webserver_interface=127.15.129.136
--webserver_port=42365
--tserver_master_addrs=127.15.129.190:34757
--never_fsync=true
--heap_profile_path=/tmp/kudu.4653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.129.136
--log_dir=/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 0d95304fa46ee5d96bcaa934c7660369f2860e06
build type DEBUG
built by None at 02 May 2024 07:43:15 UTC on 1a15035a6830
build id 85
I20240502 07:59:38.973032 4653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240502 07:59:38.974241 4653 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240502 07:59:38.982846 4658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:38.983485 4659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240502 07:59:38.984032 4661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240502 07:59:38.985054 4653 server_base.cc:1034] running on GCE node
I20240502 07:59:38.985527 4653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240502 07:59:38.986086 4653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240502 07:59:38.987231 4653 hybrid_clock.cc:648] HybridClock initialized: now 1714636778987203 us; error 40 us; skew 500 ppm
I20240502 07:59:38.989462 4653 webserver.cc:460] Webserver started at http://127.15.129.136:42365/ using document root <none> and password file <none>
I20240502 07:59:38.990015 4653 fs_manager.cc:359] Metadata directory not provided
I20240502 07:59:38.990084 4653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240502 07:59:38.993422 4653 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.003s
I20240502 07:59:38.995324 4667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:38.996332 4653 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20240502 07:59:38.996438 4653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data,/tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
uuid: "8d458f64f2f34b84bb00eea7d8d2d35a"
format_stamp: "Formatted at 2024-05-02 07:59:28 on dist-test-slave-0hx2"
I20240502 07:59:38.996850 4653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
metadata directory: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/wal
1 data directories: /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240502 07:59:39.007972 4653 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240502 07:59:39.008543 4653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240502 07:59:39.008702 4653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240502 07:59:39.009184 4653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240502 07:59:39.010149 4653 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240502 07:59:39.010197 4653 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:39.010250 4653 ts_tablet_manager.cc:590] Registered 0 tablets
I20240502 07:59:39.010280 4653 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240502 07:59:39.018878 4653 rpc_server.cc:307] RPC server started. Bound to: 127.15.129.136:41115
I20240502 07:59:39.018925 4780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.129.136:41115 every 8 connection(s)
I20240502 07:59:39.019846 4653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUnCtd5/test-tmp/rebalancer_tool-test.0.RebalancingDuringElectionStormTest.RoundRobin_0.1714636686536370-15878-0/raft_consensus-itest-cluster/ts-7/data/info.pb
I20240502 07:59:39.020017 15878 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUnCtd5/build/debug/bin/kudu as pid 4653
I20240502 07:59:39.020231 15878 test_util.cc:268] Using random seed: -64888891
I20240502 07:59:39.030671 15878 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_0 already exists
I20240502 07:59:39.030930 15878 test_util.cc:268] Using random seed: -64878194
I20240502 07:59:39.031574 4781 heartbeater.cc:347] Connected to a master server at 127.15.129.190:34757
I20240502 07:59:39.031800 4781 heartbeater.cc:464] Registering TS with master...
I20240502 07:59:39.032307 4781 heartbeater.cc:511] Master 127.15.129.190:34757 requested a full tablet report, sending...
I20240502 07:59:39.034021 2470 ts_manager.cc:194] Re-registered known tserver with Master: 8d458f64f2f34b84bb00eea7d8d2d35a (127.15.129.136:41115)
I20240502 07:59:39.034538 2470 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.129.136:50691
I20240502 07:59:39.043367 15878 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_1 already exists
I20240502 07:59:39.043529 15878 test_util.cc:268] Using random seed: -64865593
I20240502 07:59:39.047477 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "512e1322925c415f8891dfe6621160e1"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.047650 3010 raft_consensus.cc:482] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.047710 3010 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.048794 3010 raft_consensus.cc:504] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.049160 3010 leader_election.cc:288] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.049494 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.049628 2879 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.050027 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.050141 2746 raft_consensus.cc:3032] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.050182 2746 raft_consensus.cc:729] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.050339 2746 consensus_queue.cc:260] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.050647 2746 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.050684 2879 raft_consensus.cc:2454] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.051627 2746 raft_consensus.cc:2454] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.051699 2944 leader_election.cc:302] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.051846 4243 raft_consensus.cc:2781] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.051934 4243 raft_consensus.cc:686] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.052069 4243 consensus_queue.cc:237] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.052196 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.052280 3010 raft_consensus.cc:482] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.052336 3010 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.053197 3010 raft_consensus.cc:504] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.053568 3010 leader_election.cc:288] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.054219 2617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.054352 2617 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.055300 2617 raft_consensus.cc:2454] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.055364 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.055451 3010 raft_consensus.cc:482] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.055501 3010 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.055548 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.055662 2878 raft_consensus.cc:3032] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.055702 2878 raft_consensus.cc:729] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.055895 2878 consensus_queue.cc:260] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.056252 2878 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.056254 3010 raft_consensus.cc:504] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.056529 3010 leader_election.cc:288] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.056512 2470 catalog_manager.cc:5484] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 1 to 2, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 2 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.057261 2878 raft_consensus.cc:2454] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.058235 2944 leader_election.cc:302] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.058625 4243 raft_consensus.cc:2781] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.058735 3008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "236a4d787879442aaff6983243f21f85"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.058835 3008 raft_consensus.cc:482] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.058890 3008 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.059775 3008 raft_consensus.cc:504] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.060083 3008 leader_election.cc:288] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.060252 15878 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_2 already exists
I20240502 07:59:39.060400 15878 test_util.cc:268] Using random seed: -64848723
I20240502 07:59:39.058734 4243 raft_consensus.cc:686] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.060648 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.060709 4243 consensus_queue.cc:237] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.060760 2616 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.061776 2616 raft_consensus.cc:2454] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.062068 2617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.062165 2944 leader_election.cc:302] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.062176 2617 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.062477 4243 raft_consensus.cc:2781] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.062553 4243 raft_consensus.cc:686] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.062672 4243 consensus_queue.cc:237] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.063077 2617 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.063527 2944 leader_election.cc:302] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.063742 4243 raft_consensus.cc:2781] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.063819 4243 raft_consensus.cc:686] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.063925 4243 consensus_queue.cc:237] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.064074 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.064193 2878 raft_consensus.cc:3032] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.064246 2878 raft_consensus.cc:729] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.064394 2878 consensus_queue.cc:260] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.064549 2878 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.064872 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.064960 3010 raft_consensus.cc:482] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.064996 3010 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.065419 2878 raft_consensus.cc:2454] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.065723 3010 raft_consensus.cc:504] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.065765 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.065888 2746 raft_consensus.cc:3032] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.065934 2746 raft_consensus.cc:729] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.066071 2746 consensus_queue.cc:260] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.066248 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.066265 2746 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.066329 2617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.066375 2747 raft_consensus.cc:3032] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.066411 2747 raft_consensus.cc:729] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.066424 2617 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.066529 2747 consensus_queue.cc:260] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.066670 2747 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.067196 2617 raft_consensus.cc:2454] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.067234 2746 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.067495 2747 raft_consensus.cc:2454] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.067512 2944 leader_election.cc:302] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.067884 4243 raft_consensus.cc:2781] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.067983 4243 raft_consensus.cc:686] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.068089 4243 consensus_queue.cc:237] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.068089 3010 leader_election.cc:288] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.069177 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.069715 3010 raft_consensus.cc:465] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 07:59:39.071038 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.071136 3010 raft_consensus.cc:482] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.071195 3010 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.071976 2470 catalog_manager.cc:5484] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 1 to 2, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 2 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.072211 2470 catalog_manager.cc:5484] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 1 to 2, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 2 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:39.072405 2470 catalog_manager.cc:5484] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 1 to 2, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 2 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.072587 2470 catalog_manager.cc:5484] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 1 to 2, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 2 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.073145 3010 raft_consensus.cc:504] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.073784 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.073905 2878 raft_consensus.cc:3032] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.073954 2878 raft_consensus.cc:729] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.074108 2878 consensus_queue.cc:260] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.074299 2878 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.074301 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.074417 2747 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.075237 2878 raft_consensus.cc:2454] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.075439 2747 raft_consensus.cc:2454] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.075611 2944 leader_election.cc:302] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.077294 4243 raft_consensus.cc:2781] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.077461 4243 raft_consensus.cc:686] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.077584 4243 consensus_queue.cc:237] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.078804 3010 leader_election.cc:288] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.079556 2475 catalog_manager.cc:5484] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 1 to 2, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 2 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:39.080403 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "18912e0059054584a990be7ccb4a5771"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.080513 3010 raft_consensus.cc:465] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 07:59:39.081030 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f116334237934021a25a6cacae430034"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.081122 3010 raft_consensus.cc:465] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 07:59:39.082156 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.082267 3010 raft_consensus.cc:465] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 07:59:39.082952 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.083051 3010 raft_consensus.cc:482] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.083118 3010 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.084102 3010 raft_consensus.cc:504] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.084448 3010 leader_election.cc:288] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.084794 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.084929 2878 raft_consensus.cc:3032] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.085001 2878 raft_consensus.cc:729] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.085100 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.085165 2878 consensus_queue.cc:260] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.085213 2748 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.085404 2878 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.086233 2748 raft_consensus.cc:2454] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.086349 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d7b590b4226443d4899f3005840fd736"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.086385 2878 raft_consensus.cc:2454] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.086526 3010 raft_consensus.cc:482] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.086630 3010 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.086881 2946 leader_election.cc:302] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.087061 4243 raft_consensus.cc:2781] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.087146 4243 raft_consensus.cc:686] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.087641 4243 consensus_queue.cc:237] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.087940 3010 raft_consensus.cc:504] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.088294 3010 leader_election.cc:288] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.089697 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c67533803b9c496295f4f57e77a51374"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.089728 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.089820 2877 raft_consensus.cc:3032] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.089869 2877 raft_consensus.cc:729] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.089794 2475 catalog_manager.cc:5484] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 1 to 2, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 2 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:39.089900 3010 raft_consensus.cc:465] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 07:59:39.090018 2877 consensus_queue.cc:260] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.090211 2877 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.091181 2877 raft_consensus.cc:2454] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.091984 2944 leader_election.cc:302] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.092139 4243 raft_consensus.cc:2781] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.092216 4243 raft_consensus.cc:686] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.092350 4243 consensus_queue.cc:237] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.094116 3008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.094084 2475 catalog_manager.cc:5484] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 1 to 2, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 2 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.094218 3008 raft_consensus.cc:465] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 07:59:39.094496 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.094605 2748 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.095530 2748 raft_consensus.cc:2454] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.098287 3008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.098397 3008 raft_consensus.cc:482] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.098452 3008 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.099283 3008 raft_consensus.cc:504] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.099622 3008 leader_election.cc:288] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.099990 2617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.100087 2617 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.100040 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.100168 2879 raft_consensus.cc:3032] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.100211 2879 raft_consensus.cc:729] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.100395 2879 consensus_queue.cc:260] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.100555 2879 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.101047 2617 raft_consensus.cc:2454] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.101523 2879 raft_consensus.cc:2454] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.105232 2746 raft_consensus.cc:1261] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:39.105679 4243 consensus_queue.cc:1035] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.105860 2944 leader_election.cc:302] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.106282 2616 raft_consensus.cc:1261] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:39.106679 2617 raft_consensus.cc:1261] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:39.108091 2879 raft_consensus.cc:1261] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:39.108384 3008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.108487 3008 raft_consensus.cc:482] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.108538 3008 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.108636 4243 raft_consensus.cc:2781] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.108721 4243 raft_consensus.cc:686] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.108842 4243 consensus_queue.cc:237] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.109690 4524 consensus_queue.cc:1035] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.109932 4524 consensus_queue.cc:1035] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.109952 4855 consensus_queue.cc:1035] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.109997 3008 raft_consensus.cc:504] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.110267 3008 leader_election.cc:288] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.110780 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.110890 2747 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.111807 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.111883 2747 raft_consensus.cc:2454] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.111905 3010 raft_consensus.cc:482] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.111955 3010 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.112603 2946 leader_election.cc:302] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.112797 3010 raft_consensus.cc:504] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.113071 3010 leader_election.cc:288] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.113273 4524 raft_consensus.cc:2781] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.113375 4524 raft_consensus.cc:686] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.113498 4524 consensus_queue.cc:237] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.113588 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 6 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.113695 2877 raft_consensus.cc:3032] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.113734 2877 raft_consensus.cc:729] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.113867 2877 consensus_queue.cc:260] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 6, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.114014 2877 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.114959 2877 raft_consensus.cc:2454] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.115152 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.115196 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.115242 2878 raft_consensus.cc:3032] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.115294 2878 raft_consensus.cc:729] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.115310 2614 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.115412 2878 consensus_queue.cc:260] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.115595 2878 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.116250 2614 raft_consensus.cc:2454] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.116466 2878 raft_consensus.cc:2454] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.116616 2470 catalog_manager.cc:5484] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 1 to 2, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 2 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:39.117427 2944 leader_election.cc:302] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.117791 4524 raft_consensus.cc:2781] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.117863 4524 raft_consensus.cc:686] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.117970 4524 consensus_queue.cc:237] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.118126 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.118242 3010 raft_consensus.cc:482] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.118304 3010 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.119176 3010 raft_consensus.cc:504] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.119508 3010 leader_election.cc:288] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.121783 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.121905 3010 raft_consensus.cc:465] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 07:59:39.121987 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.122089 2614 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.123032 2614 raft_consensus.cc:2454] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.123664 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.123778 3010 raft_consensus.cc:482] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.123836 3010 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.123963 2944 leader_election.cc:302] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.124812 3010 raft_consensus.cc:504] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.124986 4243 raft_consensus.cc:2781] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.125056 4243 raft_consensus.cc:686] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.125172 3010 leader_election.cc:288] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.125154 4243 consensus_queue.cc:237] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.126152 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.126266 2748 raft_consensus.cc:3032] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.126305 2748 raft_consensus.cc:729] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.126432 2748 consensus_queue.cc:260] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.126617 2748 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.126665 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "811167eb170b49219b6cd22d865af334"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.126771 3010 raft_consensus.cc:482] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.126835 3010 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.127591 2748 raft_consensus.cc:2454] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.127704 3010 raft_consensus.cc:504] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.128068 2876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.128198 2876 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.128306 2470 catalog_manager.cc:5484] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 1 to 2, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 2 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:39.128562 2470 catalog_manager.cc:5484] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 1 to 2, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 2 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:39.129173 2876 raft_consensus.cc:2454] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.129419 2615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.129547 2615 raft_consensus.cc:3032] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.129601 2615 raft_consensus.cc:729] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.129773 2615 consensus_queue.cc:260] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.130071 2615 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.130761 2944 leader_election.cc:302] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.130975 4243 raft_consensus.cc:2781] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.131059 4243 raft_consensus.cc:686] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.131170 2615 raft_consensus.cc:2454] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.131196 4243 consensus_queue.cc:237] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.131588 3010 leader_election.cc:288] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.132391 2615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.132515 2615 raft_consensus.cc:3032] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.132565 2615 raft_consensus.cc:729] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.132570 15878 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_3 already exists
I20240502 07:59:39.132772 15878 test_util.cc:268] Using random seed: -64776351
I20240502 07:59:39.132728 2615 consensus_queue.cc:260] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.132902 2615 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.133124 2746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.133232 2746 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.133993 2615 raft_consensus.cc:2454] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.134172 2746 raft_consensus.cc:2454] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 2.
I20240502 07:59:39.134177 3008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.134277 3008 raft_consensus.cc:465] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Not starting forced leader election -- already a leader
I20240502 07:59:39.135192 2946 leader_election.cc:302] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.136467 4524 raft_consensus.cc:2781] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.136549 4524 raft_consensus.cc:686] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.136669 4524 consensus_queue.cc:237] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
W20240502 07:59:39.137733 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34742: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.140161 2470 catalog_manager.cc:5484] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 1 to 2, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 2 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.140446 2470 catalog_manager.cc:5484] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 1 to 2, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 2 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.140625 2470 catalog_manager.cc:5484] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 1 to 2, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 2 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:39.146407 2879 raft_consensus.cc:1261] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240502 07:59:39.146854 2747 raft_consensus.cc:1261] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 2 index: 8. (index mismatch)
I20240502 07:59:39.148440 4853 consensus_queue.cc:1035] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20240502 07:59:39.148607 4855 consensus_queue.cc:1035] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:39.195096 15878 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_4 already exists
I20240502 07:59:39.195287 15878 test_util.cc:268] Using random seed: -64713834
W20240502 07:59:39.202567 2707 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.208810 2615 raft_consensus.cc:1261] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:39.209192 4524 consensus_queue.cc:1035] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.209801 2746 raft_consensus.cc:1261] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:39.210332 4854 consensus_queue.cc:1035] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.217423 2746 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "512e1322925c415f8891dfe6621160e1"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.217595 2746 raft_consensus.cc:482] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.217655 2746 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.219662 2746 raft_consensus.cc:504] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.231876 2746 leader_election.cc:288] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 3 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.234529 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.234669 2878 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.234881 2746 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c67533803b9c496295f4f57e77a51374"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.234988 2746 raft_consensus.cc:482] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.235035 2746 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.235713 2878 raft_consensus.cc:2454] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 3.
I20240502 07:59:39.235991 2746 raft_consensus.cc:504] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.236227 2682 leader_election.cc:302] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:39.236373 2746 leader_election.cc:288] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.236392 4193 raft_consensus.cc:2781] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.236546 4193 raft_consensus.cc:686] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.236678 4193 consensus_queue.cc:237] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.238255 4835 mvcc.cc:205] Tried to move back new op lower bound from 7023152247633305600 to 7023152247300911104. Current Snapshot: MvccSnapshot[applied={T|T < 7023152247633305600}]
I20240502 07:59:39.239259 2470 catalog_manager.cc:5484] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 2 to 3, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 3 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.240918 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.241026 2878 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.242231 2878 raft_consensus.cc:2454] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.242676 2682 leader_election.cc:302] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:39.243670 2746 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.243798 2746 raft_consensus.cc:482] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.243858 2746 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.243899 4193 raft_consensus.cc:2781] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.243958 4193 raft_consensus.cc:686] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.244081 4193 consensus_queue.cc:237] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.244845 2746 raft_consensus.cc:504] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.245182 2746 leader_election.cc:288] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.246953 2615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.247067 2615 raft_consensus.cc:3032] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.247104 2615 raft_consensus.cc:729] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.247208 2615 consensus_queue.cc:260] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.247359 2615 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.248353 2615 raft_consensus.cc:2454] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.248827 2470 catalog_manager.cc:5484] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 1 to 2, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 2 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:39.252666 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.252795 3008 raft_consensus.cc:3032] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.252835 3008 raft_consensus.cc:729] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.252972 3008 consensus_queue.cc:260] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.253199 3008 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.254257 3008 raft_consensus.cc:2396] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 07:59:39.257797 2682 leader_election.cc:302] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:39.259017 2745 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.259140 2745 raft_consensus.cc:482] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.259198 2745 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.259722 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.259857 2877 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.260908 4193 raft_consensus.cc:2781] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.260919 2877 raft_consensus.cc:2454] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.260996 4193 raft_consensus.cc:686] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.261122 4193 consensus_queue.cc:237] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.263203 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.263329 3008 raft_consensus.cc:3032] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.263366 3008 raft_consensus.cc:729] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.263485 3008 consensus_queue.cc:260] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.263669 3008 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.264479 2470 catalog_manager.cc:5484] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 1 to 2, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 2 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.264660 3008 raft_consensus.cc:2454] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.268373 2745 raft_consensus.cc:504] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.268774 2745 leader_election.cc:288] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.271446 2745 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f116334237934021a25a6cacae430034"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.271581 2745 raft_consensus.cc:482] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.271649 2745 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.272652 2745 raft_consensus.cc:504] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.273375 2745 leader_election.cc:288] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.273998 2876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.274133 2876 raft_consensus.cc:3032] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.274181 2876 raft_consensus.cc:729] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.274363 2876 consensus_queue.cc:260] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.274588 2876 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.275676 2876 raft_consensus.cc:2454] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.275903 2617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.276021 2617 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.277063 2617 raft_consensus.cc:2454] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.277294 2615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.277455 2615 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.278098 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.278213 3008 raft_consensus.cc:3032] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.278259 3008 raft_consensus.cc:729] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.278296 2745 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.278359 2615 raft_consensus.cc:2454] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.278393 2745 raft_consensus.cc:482] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.278443 2745 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.278398 3008 consensus_queue.cc:260] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.278570 3008 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.279426 3008 raft_consensus.cc:2454] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.279407 2745 raft_consensus.cc:504] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.280148 2681 leader_election.cc:302] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.280639 2745 leader_election.cc:288] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:39.281250 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.281400 2747 raft_consensus.cc:482] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.281462 2747 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.281735 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.281839 3008 raft_consensus.cc:3032] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.281878 3008 raft_consensus.cc:729] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.282008 3008 consensus_queue.cc:260] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.282133 3008 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.282421 2747 raft_consensus.cc:504] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.282793 2747 leader_election.cc:288] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 3 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.283033 3008 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.283746 2745 raft_consensus.cc:1226] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Rejecting Update request from peer f1e12e7015044fbeb23a7cd6f4023bef for earlier term 2. Current term is 3. Ops: []
I20240502 07:59:39.284004 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.284184 3008 raft_consensus.cc:3032] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.284281 3008 raft_consensus.cc:729] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.284402 2681 leader_election.cc:302] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.284570 4193 raft_consensus.cc:2781] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.284767 4193 raft_consensus.cc:686] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.284893 4193 consensus_queue.cc:237] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.284535 3008 consensus_queue.cc:260] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.285218 3008 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.285507 4193 raft_consensus.cc:2781] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.285593 4193 raft_consensus.cc:686] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
W20240502 07:59:39.285598 4862 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.002s ago.
I20240502 07:59:39.285708 4193 consensus_queue.cc:237] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.286460 3008 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 3.
I20240502 07:59:39.288106 2681 leader_election.cc:302] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.288765 2470 catalog_manager.cc:5484] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 1 to 2, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 2 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.289258 2470 catalog_manager.cc:5484] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 1 to 2, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 2 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:39.289476 4906 raft_consensus.cc:2781] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.289650 4906 raft_consensus.cc:686] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.290577 4906 consensus_queue.cc:237] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.7, Last appended by leader: 7, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.291703 2742 raft_consensus.cc:1261] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:39.290699 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.291818 2747 raft_consensus.cc:482] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.291884 2747 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.292877 2747 raft_consensus.cc:504] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.293190 2747 leader_election.cc:288] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.293627 2878 raft_consensus.cc:1261] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:39.293782 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.293888 2747 raft_consensus.cc:482] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.293941 2747 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.294157 2617 raft_consensus.cc:1261] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:39.294562 4855 consensus_queue.cc:1035] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.294694 4853 consensus_queue.cc:1035] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.294793 2747 raft_consensus.cc:504] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.295079 2747 leader_election.cc:288] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.295671 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.295765 2747 raft_consensus.cc:482] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.295823 2747 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.296676 2747 raft_consensus.cc:504] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.296998 2747 leader_election.cc:288] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.297730 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.297842 2747 raft_consensus.cc:482] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.297900 2747 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.298794 2747 raft_consensus.cc:504] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.299139 2747 leader_election.cc:288] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.300446 2682 leader_election.cc:302] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:39.300777 4906 raft_consensus.cc:2781] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election won for term 2
W20240502 07:59:39.300789 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45948: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.300951 4906 raft_consensus.cc:686] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.301079 4906 consensus_queue.cc:237] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.301355 2875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 13 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.301489 2742 raft_consensus.cc:1226] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Rejecting Update request from peer c93a6d22180a41a686c4556afd6a3e6b for earlier term 1. Current term is 2. Ops: []
I20240502 07:59:39.301491 2875 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.302134 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 13 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.302191 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.302255 2614 raft_consensus.cc:3032] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.302296 2613 raft_consensus.cc:3032] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.302348 2613 raft_consensus.cc:729] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.302417 2617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.302513 2617 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.302673 2875 raft_consensus.cc:2396] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 2 because replica has last-logged OpId of term: 1 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 13.
I20240502 07:59:39.302678 2612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.302760 2612 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.302898 2874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.302994 2874 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.303588 2617 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 3.
I20240502 07:59:39.303735 2612 raft_consensus.cc:2454] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.303938 2874 raft_consensus.cc:2454] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.302482 2613 consensus_queue.cc:260] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.304219 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.304325 2613 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.301355 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.304441 2879 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.305214 2613 raft_consensus.cc:2454] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.305341 2879 raft_consensus.cc:2454] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.304330 2616 raft_consensus.cc:3032] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.305531 2616 raft_consensus.cc:729] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.305667 2616 consensus_queue.cc:260] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.302304 2614 raft_consensus.cc:729] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.305838 2616 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.305866 2614 consensus_queue.cc:260] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 1.14, Last appended by leader: 14, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.306149 2614 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.306771 2616 raft_consensus.cc:2454] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.307106 2614 raft_consensus.cc:2396] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 2 because replica has last-logged OpId of term: 1 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 13.
I20240502 07:59:39.307225 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 5 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.307332 2878 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.307747 2876 raft_consensus.cc:1226] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Rejecting Update request from peer c93a6d22180a41a686c4556afd6a3e6b for earlier term 1. Current term is 2. Ops: []
I20240502 07:59:39.308151 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.308269 2747 raft_consensus.cc:482] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.308285 2878 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.308331 2747 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.309711 2470 catalog_manager.cc:5484] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 2 to 3, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 3 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.310503 2682 leader_election.cc:302] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:39.310689 2682 leader_election.cc:302] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:39.310739 4906 raft_consensus.cc:2781] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.310843 4193 raft_consensus.cc:2781] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.310909 4906 raft_consensus.cc:686] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.311062 4193 raft_consensus.cc:686] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.311121 4906 consensus_queue.cc:237] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.311223 4193 consensus_queue.cc:237] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.311691 2682 leader_election.cc:302] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:39.311981 4193 raft_consensus.cc:2781] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.312050 4193 raft_consensus.cc:686] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.312153 4193 consensus_queue.cc:237] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.312067 2682 leader_election.cc:302] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1; no voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:39.312721 4193 raft_consensus.cc:2726] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20240502 07:59:39.314357 4243 consensus_queue.cc:1035] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.314585 2745 raft_consensus.cc:1226] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Rejecting Update request from peer c93a6d22180a41a686c4556afd6a3e6b for earlier term 1. Current term is 2. Ops: [1.14-1.14]
I20240502 07:59:39.315498 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.315605 3008 raft_consensus.cc:3032] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.315656 3008 raft_consensus.cc:729] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.315766 3008 consensus_queue.cc:260] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.315850 2613 raft_consensus.cc:1261] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 3 index: 9. (index mismatch)
I20240502 07:59:39.315939 3008 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.316233 4193 consensus_queue.cc:1035] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:39.316926 3008 raft_consensus.cc:2454] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.320432 4812 mvcc.cc:205] Tried to move back new op lower bound from 7023152247966547968 to 7023152247163564032. Current Snapshot: MvccSnapshot[applied={T|T < 7023152247966547968}]
I20240502 07:59:39.321761 2742 raft_consensus.cc:1261] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:39.321906 15878 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_5 already exists
I20240502 07:59:39.322053 15878 test_util.cc:268] Using random seed: -64587069
I20240502 07:59:39.312731 2747 raft_consensus.cc:504] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.322947 2744 raft_consensus.cc:1226] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Rejecting Update request from peer f1e12e7015044fbeb23a7cd6f4023bef for earlier term 2. Current term is 3. Ops: [2.15-2.15]
I20240502 07:59:39.323765 2747 leader_election.cc:288] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 3 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.324234 2475 catalog_manager.cc:5484] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 1 to 2, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 2 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.324474 2744 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.324477 2475 catalog_manager.cc:5484] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 1 to 2, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 2 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.324563 2744 raft_consensus.cc:482] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.324613 2744 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.324672 2475 catalog_manager.cc:5484] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 1 to 2, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 2 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.325595 2744 raft_consensus.cc:504] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.325661 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.325768 3010 raft_consensus.cc:3032] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.325805 3010 raft_consensus.cc:729] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.325906 2744 leader_election.cc:288] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.325923 3010 consensus_queue.cc:260] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 2.16, Last appended by leader: 16, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.326027 3010 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.326334 3008 raft_consensus.cc:1261] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 3 index: 9. (index mismatch)
I20240502 07:59:39.326674 2744 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.326759 2744 raft_consensus.cc:482] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.326797 3010 raft_consensus.cc:2396] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 3 because replica has last-logged OpId of term: 2 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 14.
I20240502 07:59:39.326802 2744 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.327570 2744 raft_consensus.cc:504] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.327647 2742 raft_consensus.cc:1226] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Rejecting Update request from peer f1e12e7015044fbeb23a7cd6f4023bef for earlier term 1. Current term is 2. Ops: []
I20240502 07:59:39.327834 2744 leader_election.cc:288] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.328172 4906 consensus_queue.cc:1035] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:39.329155 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.329192 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.329304 2879 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.329597 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 14 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.329697 2878 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.329787 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.329883 2613 raft_consensus.cc:3032] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.329917 2613 raft_consensus.cc:729] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.330039 2613 consensus_queue.cc:260] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.330175 2613 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.330224 2879 raft_consensus.cc:2454] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.330395 2744 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.330520 2744 raft_consensus.cc:482] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.330572 2744 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.331302 2613 raft_consensus.cc:2454] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.331395 2744 raft_consensus.cc:504] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.331707 2744 leader_election.cc:288] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.331723 2682 leader_election.cc:302] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:39.332120 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.332219 2879 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.332548 4915 raft_consensus.cc:2781] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.332672 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.332749 2747 raft_consensus.cc:482] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.332792 2747 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
W20240502 07:59:39.332859 2969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33724: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.333719 2747 raft_consensus.cc:504] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.334149 2747 leader_election.cc:288] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 3 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.334040 2475 catalog_manager.cc:5484] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 1 to 2, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 2 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.334592 2879 raft_consensus.cc:2454] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.334591 4915 raft_consensus.cc:686] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.334827 2878 raft_consensus.cc:2396] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 3 because replica has last-logged OpId of term: 2 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 14.
I20240502 07:59:39.335072 2682 leader_election.cc:302] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:39.335220 2682 leader_election.cc:302] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1; no voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:39.335350 4193 raft_consensus.cc:2781] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.335315 4885 consensus_queue.cc:1035] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.335402 4193 raft_consensus.cc:686] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.335500 4193 consensus_queue.cc:237] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.335844 4885 consensus_queue.cc:1046] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: INVALID_TERM, Last received: 1.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:39.335922 4193 raft_consensus.cc:2726] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20240502 07:59:39.336169 4853 raft_consensus.cc:3032] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.336217 4853 raft_consensus.cc:729] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.336342 4853 consensus_queue.cc:260] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.336649 4853 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.336959 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.337046 2747 raft_consensus.cc:482] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.338377 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 10 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.338487 3009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.338562 3010 raft_consensus.cc:2396] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 2 because replica has last-logged OpId of term: 1 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 10.
I20240502 07:59:39.338572 3009 raft_consensus.cc:3032] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.338634 3009 raft_consensus.cc:729] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.338786 3009 consensus_queue.cc:260] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.338941 3009 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.339040 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.339140 2613 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.339807 3009 raft_consensus.cc:2454] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.340129 2613 raft_consensus.cc:2454] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 3.
I20240502 07:59:39.344354 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.344503 3008 raft_consensus.cc:3032] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.344547 3008 raft_consensus.cc:729] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.344661 3008 consensus_queue.cc:260] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.344841 3008 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.345482 2681 leader_election.cc:302] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1; no voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:39.346138 4920 raft_consensus.cc:2726] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20240502 07:59:39.346321 3008 raft_consensus.cc:2454] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 3.
I20240502 07:59:39.346671 2681 leader_election.cc:302] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.347767 4920 raft_consensus.cc:2781] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.347841 4920 raft_consensus.cc:686] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.347956 4920 consensus_queue.cc:237] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
W20240502 07:59:39.348645 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60740: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
W20240502 07:59:39.354087 2969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
W20240502 07:59:39.354120 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34742: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.337092 2747 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.355183 2747 raft_consensus.cc:504] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.355841 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.355952 3008 raft_consensus.cc:3032] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.355998 3008 raft_consensus.cc:729] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.356169 3008 consensus_queue.cc:260] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.356316 3008 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.356746 4915 consensus_queue.cc:237] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.357250 3008 raft_consensus.cc:2396] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 07:59:39.357260 2747 leader_election.cc:288] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 3 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:39.358273 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d7b590b4226443d4899f3005840fd736"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.358306 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.358390 2742 raft_consensus.cc:482] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.358423 2877 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.358458 2742 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.359378 2742 raft_consensus.cc:504] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.359465 2877 raft_consensus.cc:2454] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 3.
I20240502 07:59:39.359743 2742 leader_election.cc:288] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 3 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
W20240502 07:59:39.360253 2969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33724: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.360954 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.361052 2742 raft_consensus.cc:482] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.361101 2742 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.362032 2742 raft_consensus.cc:504] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.362162 2614 raft_consensus.cc:1261] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:39.362391 2742 leader_election.cc:288] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 3 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.362655 2877 raft_consensus.cc:1261] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:39.362754 4243 consensus_queue.cc:1035] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.363356 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.363451 2742 raft_consensus.cc:482] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.363505 2742 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.363585 4243 consensus_queue.cc:1035] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.363739 2475 catalog_manager.cc:5484] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 1 to 2, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 2 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.363970 2475 catalog_manager.cc:5484] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 1 to 2, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 2 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.364437 2742 raft_consensus.cc:504] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.364744 2742 leader_election.cc:288] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.366484 3009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.366595 3009 raft_consensus.cc:3032] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.366642 3009 raft_consensus.cc:729] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.366786 3009 consensus_queue.cc:260] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.366899 3009 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.367934 3009 raft_consensus.cc:2454] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 3.
W20240502 07:59:39.368641 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60740: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.368978 3009 raft_consensus.cc:1261] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 07:59:39.369243 2681 leader_election.cc:302] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.369431 4915 raft_consensus.cc:2781] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.369556 4915 raft_consensus.cc:686] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.369669 4915 consensus_queue.cc:237] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.370375 2745 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.370501 2745 raft_consensus.cc:482] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.370550 2745 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.371456 2745 raft_consensus.cc:504] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.371637 4862 mvcc.cc:205] Tried to move back new op lower bound from 7023152248263794688 to 7023152247268466688. Current Snapshot: MvccSnapshot[applied={T|T < 7023152248263794688 or (T in {7023152248263794688})}]
I20240502 07:59:39.371776 2745 leader_election.cc:288] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
W20240502 07:59:39.372027 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45936: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.372292 2748 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "811167eb170b49219b6cd22d865af334"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.372387 2748 raft_consensus.cc:482] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.372438 2748 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.373315 2748 raft_consensus.cc:504] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.373281 2470 catalog_manager.cc:5484] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 2 to 3, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 3 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.373651 2748 leader_election.cc:288] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 3 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.374444 3009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.374557 3009 raft_consensus.cc:3032] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.374595 3009 raft_consensus.cc:729] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.374706 3009 consensus_queue.cc:260] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.374835 3009 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.375797 3009 raft_consensus.cc:2396] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 07:59:39.377460 2748 raft_consensus.cc:1226] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Rejecting Update request from peer f1e12e7015044fbeb23a7cd6f4023bef for earlier term 2. Current term is 3. Ops: [2.7-2.7]
I20240502 07:59:39.377806 4853 consensus_queue.cc:1046] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: INVALID_TERM, Last received: 2.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:39.378038 4853 raft_consensus.cc:3032] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.378082 4853 raft_consensus.cc:729] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.378186 4853 consensus_queue.cc:260] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.378326 4853 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
W20240502 07:59:39.379664 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34742: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.380414 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.380535 3008 raft_consensus.cc:2396] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 3 because replica has last-logged OpId of term: 2 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 07:59:39.380700 4915 consensus_queue.cc:1035] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:39.381563 2745 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:39.381659 2745 raft_consensus.cc:482] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.381711 2745 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.382227 2877 raft_consensus.cc:1261] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:39.382578 4854 consensus_queue.cc:1035] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.383671 2878 raft_consensus.cc:1261] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 2 index: 7. (index mismatch)
I20240502 07:59:39.384054 2745 raft_consensus.cc:504] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.385193 2876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.385303 2876 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.385977 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.386080 2878 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.386679 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.386802 2616 raft_consensus.cc:3032] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.386844 2616 raft_consensus.cc:729] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.386952 2615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.386988 2878 raft_consensus.cc:2454] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.386976 2616 consensus_queue.cc:260] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.387055 2615 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
W20240502 07:59:39.385982 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34756: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.387144 2616 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.387434 2612 raft_consensus.cc:1261] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:39.387791 3009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.387884 3009 raft_consensus.cc:3032] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.387928 3009 raft_consensus.cc:729] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.388072 2616 raft_consensus.cc:2454] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.388172 2615 raft_consensus.cc:2454] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 3.
I20240502 07:59:39.388382 3009 consensus_queue.cc:260] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.388521 3009 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.389382 2876 raft_consensus.cc:2396] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 3 because replica has last-logged OpId of term: 2 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 07:59:39.389662 3009 raft_consensus.cc:2396] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 07:59:39.390278 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.390383 2879 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.390349 4853 consensus_queue.cc:1035] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.391373 2879 raft_consensus.cc:2454] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.392014 2615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.392127 2615 raft_consensus.cc:3032] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.392179 2615 raft_consensus.cc:729] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.392323 2615 consensus_queue.cc:260] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.392505 2615 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.393563 2615 raft_consensus.cc:2454] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 2.
I20240502 07:59:39.397023 2682 leader_election.cc:302] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters: f1e12e7015044fbeb23a7cd6f4023bef
W20240502 07:59:39.397166 2969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33724: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.397642 2745 leader_election.cc:288] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 3 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.397800 4906 raft_consensus.cc:2781] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.397871 4906 raft_consensus.cc:686] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.397989 4906 consensus_queue.cc:237] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.398285 4193 consensus_queue.cc:1035] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:39.398869 2682 leader_election.cc:302] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:39.399020 2682 leader_election.cc:302] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1; no voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:39.399143 4906 raft_consensus.cc:2781] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.399214 4906 raft_consensus.cc:686] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.399283 4193 raft_consensus.cc:2726] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20240502 07:59:39.399322 2470 catalog_manager.cc:5484] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 2 to 3, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 3 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.399405 4906 consensus_queue.cc:237] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.399675 2617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.399783 2617 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.400213 2682 leader_election.cc:302] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:39.400416 4915 raft_consensus.cc:2781] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.400478 4915 raft_consensus.cc:686] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.400589 4915 consensus_queue.cc:237] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.400833 2617 raft_consensus.cc:2454] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 3.
I20240502 07:59:39.401602 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.401726 2878 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.402770 2878 raft_consensus.cc:2454] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 3.
W20240502 07:59:39.408926 2969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
W20240502 07:59:39.410007 2707 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60740: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.412009 2682 leader_election.cc:302] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters: f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:39.412346 4920 raft_consensus.cc:2781] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.412427 4920 raft_consensus.cc:686] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.412554 4920 consensus_queue.cc:237] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.413156 2877 raft_consensus.cc:1261] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 07:59:39.413420 2682 leader_election.cc:302] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters: f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:39.413677 4922 raft_consensus.cc:2781] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.413786 4922 raft_consensus.cc:686] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.413834 2467 catalog_manager.cc:5484] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 2 to 3, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 3 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:39.413918 4922 consensus_queue.cc:237] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.414079 2467 catalog_manager.cc:5484] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 1 to 2, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 2 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.414248 2467 catalog_manager.cc:5484] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 1 to 2, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 2 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:39.414575 4922 consensus_queue.cc:1035] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:39.415047 2617 raft_consensus.cc:1261] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20240502 07:59:39.418469 2475 catalog_manager.cc:5484] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 2 to 3, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 3 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.418704 2475 catalog_manager.cc:5484] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 2 to 3, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 3 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:39.420876 15878 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_6 already exists
I20240502 07:59:39.421077 15878 test_util.cc:268] Using random seed: -64488045
W20240502 07:59:39.421528 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34742: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.421839 3008 raft_consensus.cc:1261] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20240502 07:59:39.422287 4922 consensus_queue.cc:1035] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:39.427618 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "18912e0059054584a990be7ccb4a5771"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.427769 2613 raft_consensus.cc:482] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.427845 2613 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.428959 2613 raft_consensus.cc:504] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.429497 2613 leader_election.cc:288] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 2 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.429648 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.429783 2877 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.430955 2877 raft_consensus.cc:2454] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 2.
I20240502 07:59:39.431730 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.431850 3008 raft_consensus.cc:3032] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.431903 3008 raft_consensus.cc:729] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.432041 3008 consensus_queue.cc:260] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.432200 3008 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.433274 3008 raft_consensus.cc:2454] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 2.
I20240502 07:59:39.433509 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f116334237934021a25a6cacae430034"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.433624 2613 raft_consensus.cc:482] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.433688 2613 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.434237 2550 leader_election.cc:302] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.434456 4074 raft_consensus.cc:2781] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.434542 4074 raft_consensus.cc:686] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 2 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.434623 2613 raft_consensus.cc:504] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.434689 4074 consensus_queue.cc:237] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
W20240502 07:59:39.434908 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45926: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.434942 2613 leader_election.cc:288] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.435927 4920 consensus_queue.cc:1035] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20240502 07:59:39.436925 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.437033 2745 raft_consensus.cc:3032] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.437069 2745 raft_consensus.cc:729] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.437192 2745 consensus_queue.cc:260] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.437350 2745 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.437786 2470 catalog_manager.cc:5484] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 1 to 2, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 2 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.438417 2745 raft_consensus.cc:2396] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 07:59:39.440858 15878 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_7 already exists
I20240502 07:59:39.441017 15878 test_util.cc:268] Using random seed: -64468103
W20240502 07:59:39.444059 2969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33724: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
W20240502 07:59:39.449064 2969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33736: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
W20240502 07:59:39.453785 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34756: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.454820 2878 raft_consensus.cc:1261] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20240502 07:59:39.455246 4915 consensus_queue.cc:1035] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:39.455969 4386 heartbeater.cc:503] Master 127.15.129.190:34757 was elected leader, sending a full tablet report...
I20240502 07:59:39.456530 2617 raft_consensus.cc:1261] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 7. (index mismatch)
I20240502 07:59:39.456668 2615 raft_consensus.cc:1261] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 07:59:39.457132 2614 raft_consensus.cc:1261] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20240502 07:59:39.457430 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.457542 3010 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.458698 3010 raft_consensus.cc:2454] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
W20240502 07:59:39.458940 2707 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60744: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.459098 3010 raft_consensus.cc:1261] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20240502 07:59:39.459113 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60740: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
W20240502 07:59:39.461544 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45962: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.462767 2550 leader_election.cc:302] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters: e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:39.462873 4920 consensus_queue.cc:1035] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.462998 4920 consensus_queue.cc:1225] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Peer f1e12e7015044fbeb23a7cd6f4023bef log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 07:59:39.464293 4847 raft_consensus.cc:2781] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.464395 4847 raft_consensus.cc:686] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.464542 4847 consensus_queue.cc:237] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.468410 2878 raft_consensus.cc:1261] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 07:59:39.468694 3010 pending_rounds.cc:76] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef: Aborting all ops after (but not including) 3
I20240502 07:59:39.468758 3010 pending_rounds.cc:97] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 07:59:39.468825 3010 raft_consensus.cc:2866] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 07:59:39.468820 4915 consensus_queue.cc:1035] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:39.468856 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33736: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.469499 2475 catalog_manager.cc:5484] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 2 to 3, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 3 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
W20240502 07:59:39.471221 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34742: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.474514 3010 raft_consensus.cc:1261] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 07:59:39.475574 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.475703 2613 raft_consensus.cc:482] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.475747 2613 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.475747 2879 raft_consensus.cc:1261] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:39.476163 4915 consensus_queue.cc:1035] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.476740 2613 raft_consensus.cc:504] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.477105 2613 leader_election.cc:288] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.477563 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.477675 2879 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.478212 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.478338 2745 raft_consensus.cc:3032] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.478412 2745 raft_consensus.cc:729] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.478603 2879 raft_consensus.cc:2454] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
I20240502 07:59:39.478580 2745 consensus_queue.cc:260] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.478734 2745 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.478927 2551 leader_election.cc:302] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:39.479106 4074 raft_consensus.cc:2781] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.479192 4074 raft_consensus.cc:686] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.479336 4074 consensus_queue.cc:237] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.479734 2745 raft_consensus.cc:2396] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
W20240502 07:59:39.480690 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34756: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.481752 2475 catalog_manager.cc:5484] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 2 to 3, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 3 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.484669 2613 raft_consensus.cc:1261] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20240502 07:59:39.486959 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33724: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.487016 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.487125 2616 raft_consensus.cc:482] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.487179 2616 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.487761 2879 raft_consensus.cc:1261] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:39.488234 2616 raft_consensus.cc:504] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.488384 4883 consensus_queue.cc:1035] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:39.488531 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60744: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.488610 2616 leader_election.cc:288] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:39.488624 2613 raft_consensus.cc:1261] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:39.488929 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.489048 2879 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.489267 4922 consensus_queue.cc:1035] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:39.489341 4193 consensus_queue.cc:1035] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.489570 4922 consensus_queue.cc:1035] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:39.490082 2879 raft_consensus.cc:2454] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
I20240502 07:59:39.490056 4193 consensus_queue.cc:1035] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.490501 4915 consensus_queue.cc:1035] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.490657 4915 consensus_queue.cc:1225] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Peer f1e12e7015044fbeb23a7cd6f4023bef log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 07:59:39.491015 2551 leader_election.cc:302] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:39.491174 4847 raft_consensus.cc:2781] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.491243 4847 raft_consensus.cc:686] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.491382 4847 consensus_queue.cc:237] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.492427 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.492537 3008 raft_consensus.cc:3032] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.492578 3008 raft_consensus.cc:729] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.492699 3008 consensus_queue.cc:260] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.492843 3008 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.493649 3010 pending_rounds.cc:76] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef: Aborting all ops after (but not including) 3
I20240502 07:59:39.493744 3010 pending_rounds.cc:97] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 07:59:39.493837 3010 raft_consensus.cc:2866] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 07:59:39.493752 3008 raft_consensus.cc:2396] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 07:59:39.496147 4855 consensus_queue.cc:1035] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.496599 2475 catalog_manager.cc:5484] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 2 to 3, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 3 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.499275 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.499415 2613 raft_consensus.cc:482] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.499469 2613 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
W20240502 07:59:39.500254 2969 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33736: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.501924 2613 raft_consensus.cc:504] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
W20240502 07:59:39.502149 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60740: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.502372 2613 leader_election.cc:288] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 4 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.504019 2616 raft_consensus.cc:1226] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Rejecting Update request from peer e0a355821af5483f96aec7e128cc18e1 for earlier term 3. Current term is 4. Ops: [3.6-3.8]
I20240502 07:59:39.506057 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.506170 2616 raft_consensus.cc:482] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.506222 2616 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.506449 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.506583 3010 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.507236 2616 raft_consensus.cc:504] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.507603 2616 leader_election.cc:288] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.507740 3010 raft_consensus.cc:2396] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 4 because replica has last-logged OpId of term: 3 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 07:59:39.508729 2876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.508855 2876 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.510978 4906 consensus_queue.cc:1046] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:39.511150 4906 raft_consensus.cc:3032] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.511191 4906 raft_consensus.cc:729] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.511343 4906 consensus_queue.cc:260] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.511469 2876 raft_consensus.cc:2454] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
I20240502 07:59:39.511504 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.511530 4906 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.511781 2551 leader_election.cc:302] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:39.511947 4074 raft_consensus.cc:2781] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.511922 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.512025 4074 raft_consensus.cc:686] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.512041 2748 raft_consensus.cc:3032] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.512082 2748 raft_consensus.cc:729] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.512208 2748 consensus_queue.cc:260] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.512169 4074 consensus_queue.cc:237] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.512333 2748 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.513170 2615 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "236a4d787879442aaff6983243f21f85"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.513273 2615 raft_consensus.cc:482] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.513348 2615 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.513361 2748 raft_consensus.cc:2396] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 07:59:39.513944 2745 raft_consensus.cc:2396] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 4 because replica has last-logged OpId of term: 3 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 07:59:39.514281 2615 raft_consensus.cc:504] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.514659 2615 leader_election.cc:288] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:39.514761 2470 catalog_manager.cc:5484] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 2 to 3, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 3 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
W20240502 07:59:39.515087 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34756: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.515753 2876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 16 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.515866 2876 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.516803 2876 raft_consensus.cc:2454] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
I20240502 07:59:39.517067 2553 leader_election.cc:302] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b; no voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:39.518206 4847 raft_consensus.cc:2726] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 07:59:39.517077 2551 leader_election.cc:302] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:39.518687 4847 raft_consensus.cc:2781] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.518764 4847 raft_consensus.cc:686] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.518887 4847 consensus_queue.cc:237] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 2.16, Last appended by leader: 16, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.520273 3009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 16 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.520396 3009 raft_consensus.cc:3032] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.520444 3009 raft_consensus.cc:729] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.520583 3009 consensus_queue.cc:260] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 2.16, Last appended by leader: 16, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.520783 3009 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.520725 2470 catalog_manager.cc:5484] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 2 to 3, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 3 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
W20240502 07:59:39.521580 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34742: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.521780 3009 raft_consensus.cc:2454] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
W20240502 07:59:39.522866 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33698: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.523507 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.523633 2616 raft_consensus.cc:482] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.523689 2616 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.524627 2616 raft_consensus.cc:504] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.525681 2616 leader_election.cc:288] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 4 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
W20240502 07:59:39.526572 2707 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60744: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.526536 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.526950 2745 raft_consensus.cc:3032] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.527007 2745 raft_consensus.cc:729] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.527146 2745 consensus_queue.cc:260] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.527271 2745 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.528210 2745 raft_consensus.cc:2396] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 07:59:39.528311 3009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.528424 3009 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.529359 3009 raft_consensus.cc:2454] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 4.
I20240502 07:59:39.529593 2550 leader_election.cc:302] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.529871 4847 raft_consensus.cc:2781] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:39.529955 4847 raft_consensus.cc:686] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 4 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.530064 4847 consensus_queue.cc:237] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.530618 2878 raft_consensus.cc:1261] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 16. Preceding OpId from leader: term: 3 index: 18. (index mismatch)
I20240502 07:59:39.531316 2614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.531327 4910 consensus_queue.cc:1035] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 07:59:39.531422 2614 raft_consensus.cc:482] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.531453 2878 raft_consensus.cc:1261] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 07:59:39.531474 2614 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.532454 2614 raft_consensus.cc:504] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.532773 2614 leader_election.cc:288] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.533682 4910 consensus_queue.cc:1035] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.534600 2615 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.534703 2615 raft_consensus.cc:482] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.534754 2615 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.535439 4839 mvcc.cc:205] Tried to move back new op lower bound from 7023152248952897536 to 7023152248910671872. Current Snapshot: MvccSnapshot[applied={T|T < 7023152248952897536}]
I20240502 07:59:39.535666 2615 raft_consensus.cc:504] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.536191 3008 raft_consensus.cc:1261] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 16. Preceding OpId from leader: term: 3 index: 18. (index mismatch)
I20240502 07:59:39.536566 3008 raft_consensus.cc:1261] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 07:59:39.538530 4847 consensus_queue.cc:1035] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 07:59:39.539121 4847 consensus_queue.cc:1035] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.539212 4847 consensus_queue.cc:1225] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Peer f1e12e7015044fbeb23a7cd6f4023bef log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20240502 07:59:39.540084 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33724: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.540210 2475 catalog_manager.cc:5484] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 3 to 4, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 4 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
W20240502 07:59:39.541265 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33736: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.542450 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 14 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.542564 2877 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.542943 2876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.543030 2876 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.543593 2877 raft_consensus.cc:2454] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
I20240502 07:59:39.544071 2876 raft_consensus.cc:2454] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
I20240502 07:59:39.544180 2551 leader_election.cc:302] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:39.544405 4074 raft_consensus.cc:2781] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.544446 2551 leader_election.cc:302] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:39.544485 4074 raft_consensus.cc:686] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.544620 4905 raft_consensus.cc:2781] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.544719 4905 raft_consensus.cc:686] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.544636 4074 consensus_queue.cc:237] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 1.14, Last appended by leader: 14, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.544778 3008 pending_rounds.cc:76] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef: Aborting all ops after (but not including) 3
I20240502 07:59:39.544823 3008 pending_rounds.cc:97] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 07:59:39.544821 4905 consensus_queue.cc:237] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.544853 3008 raft_consensus.cc:2866] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 07:59:39.545406 15878 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_8 already exists
I20240502 07:59:39.545570 15878 test_util.cc:268] Using random seed: -64363553
I20240502 07:59:39.546527 2615 leader_election.cc:288] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.548856 2614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.548976 2614 raft_consensus.cc:482] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.549047 2614 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.550611 2614 raft_consensus.cc:504] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.550967 2614 leader_election.cc:288] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.551330 2876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.551440 2876 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.551496 2614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.551606 2614 raft_consensus.cc:482] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.551661 2614 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.552369 2876 raft_consensus.cc:2454] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
I20240502 07:59:39.552500 2614 raft_consensus.cc:504] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.552778 2551 leader_election.cc:302] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:39.552891 2614 leader_election.cc:288] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.553007 4905 raft_consensus.cc:2781] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.553085 4905 raft_consensus.cc:686] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.553210 4905 consensus_queue.cc:237] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.553406 2617 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.553504 2617 raft_consensus.cc:482] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.553567 2617 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.554435 2617 raft_consensus.cc:504] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.554775 2617 leader_election.cc:288] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 2 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.554821 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.554961 2745 raft_consensus.cc:3032] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.555009 2745 raft_consensus.cc:729] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.555155 2745 consensus_queue.cc:260] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.555294 2745 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.556274 2745 raft_consensus.cc:2396] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 07:59:39.556638 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.556737 2616 raft_consensus.cc:482] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.556785 2616 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.557221 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 14 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.557334 2745 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
W20240502 07:59:39.557479 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34756: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.557760 2616 raft_consensus.cc:504] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.558301 2616 leader_election.cc:288] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.558948 2876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.559046 2876 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.559587 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.559672 2616 raft_consensus.cc:482] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.559720 2616 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.559953 2876 raft_consensus.cc:2454] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
I20240502 07:59:39.560343 2551 leader_election.cc:302] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:39.560514 4847 raft_consensus.cc:2781] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.560536 2616 raft_consensus.cc:504] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.560590 4847 raft_consensus.cc:686] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.560619 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.560714 4847 consensus_queue.cc:237] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
W20240502 07:59:39.560953 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60740: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.562325 3008 raft_consensus.cc:2454] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 2.
W20240502 07:59:39.562400 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.562467 2616 leader_election.cc:288] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 4 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.562541 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.562644 3010 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.562677 2876 raft_consensus.cc:1261] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 07:59:39.563078 2745 raft_consensus.cc:2454] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
I20240502 07:59:39.563372 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.563462 2745 raft_consensus.cc:3032] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.563495 2745 raft_consensus.cc:729] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.563443 2467 catalog_manager.cc:5484] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:39.563661 3010 raft_consensus.cc:2454] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
I20240502 07:59:39.563663 2745 consensus_queue.cc:260] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.563817 2745 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.563742 2467 catalog_manager.cc:5484] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 2 to 3, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 3 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.563915 2467 catalog_manager.cc:5484] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 2 to 3, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 3 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.564343 2550 leader_election.cc:302] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.564507 4847 raft_consensus.cc:2781] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.564577 4847 raft_consensus.cc:686] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 2 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.564690 4847 consensus_queue.cc:237] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.564849 2745 raft_consensus.cc:2454] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
I20240502 07:59:39.565980 4847 consensus_queue.cc:1035] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.566176 2550 leader_election.cc:302] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.566393 4847 raft_consensus.cc:2781] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.566466 4847 raft_consensus.cc:686] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.566572 4847 consensus_queue.cc:237] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.567243 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.567411 2745 raft_consensus.cc:3032] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.567492 2745 raft_consensus.cc:729] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.567696 2745 consensus_queue.cc:260] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.567888 2745 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.567965 2466 catalog_manager.cc:5484] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 2 to 3, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 3 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.568212 2466 catalog_manager.cc:5484] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 1 to 2, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 2 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.568768 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 14 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.568889 3010 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
W20240502 07:59:39.569212 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.569536 2745 raft_consensus.cc:2396] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 07:59:39.569722 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.569844 2748 raft_consensus.cc:2379] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b in current term 2: Already voted for candidate e0a355821af5483f96aec7e128cc18e1 in this term.
I20240502 07:59:39.569924 3010 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 4.
I20240502 07:59:39.570092 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.570166 2748 raft_consensus.cc:3032] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.570196 2748 raft_consensus.cc:729] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.570183 2550 leader_election.cc:302] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.570323 2748 consensus_queue.cc:260] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.570353 4905 raft_consensus.cc:2781] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:39.570482 2748 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.570590 4905 raft_consensus.cc:686] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 4 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.570722 4905 consensus_queue.cc:237] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 3.14, Last appended by leader: 14, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.571326 2748 raft_consensus.cc:2396] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20240502 07:59:39.572237 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60744: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.572705 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.573305 2748 raft_consensus.cc:1261] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 07:59:39.574018 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 14 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.574105 2748 raft_consensus.cc:3032] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.574136 2748 raft_consensus.cc:729] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.574261 2748 consensus_queue.cc:260] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 3.14, Last appended by leader: 14, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.574249 2466 catalog_manager.cc:5484] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 2 to 3, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 3 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.574384 2748 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.574478 2466 catalog_manager.cc:5484] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 3 to 4, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 4 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
W20240502 07:59:39.574348 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45948: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.575209 2748 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 4.
W20240502 07:59:39.576082 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60754: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.576215 4847 consensus_queue.cc:1035] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.576308 4847 consensus_queue.cc:1225] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Peer e0a355821af5483f96aec7e128cc18e1 log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 07:59:39.578398 2616 raft_consensus.cc:482] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.578480 2616 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.579502 2616 raft_consensus.cc:504] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.584409 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.584565 2878 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.585031 2616 leader_election.cc:288] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:39.585727 2878 raft_consensus.cc:2454] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
I20240502 07:59:39.585753 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
W20240502 07:59:39.585860 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34742: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.585994 3010 raft_consensus.cc:3032] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.586094 3010 raft_consensus.cc:729] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.586294 3010 consensus_queue.cc:260] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.586585 3010 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
W20240502 07:59:39.586633 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33752: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.587155 2551 leader_election.cc:302] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:39.587404 4910 raft_consensus.cc:2781] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.587481 4910 raft_consensus.cc:686] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.587615 4910 consensus_queue.cc:237] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.587719 3010 raft_consensus.cc:2454] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
I20240502 07:59:39.588289 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.588407 2616 raft_consensus.cc:482] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.588469 2616 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.589407 2616 raft_consensus.cc:504] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.589725 2616 leader_election.cc:288] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
W20240502 07:59:39.590869 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.590790 2475 catalog_manager.cc:5484] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 2 to 3, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 3 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:39.594518 2613 raft_consensus.cc:1226] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Rejecting Update request from peer f1e12e7015044fbeb23a7cd6f4023bef for earlier term 2. Current term is 3. Ops: [2.7-2.7]
W20240502 07:59:39.594887 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33736: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.594933 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.595047 2613 raft_consensus.cc:482] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.595113 2613 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.595255 4883 consensus_queue.cc:1046] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: INVALID_TERM, Last received: 2.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:39.595459 4883 raft_consensus.cc:3032] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.595510 4883 raft_consensus.cc:729] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.595640 4883 consensus_queue.cc:260] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.595793 4883 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.596170 2613 raft_consensus.cc:504] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.596195 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.596525 2613 leader_election.cc:288] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.599412 3009 raft_consensus.cc:1261] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20240502 07:59:39.599519 2876 raft_consensus.cc:1261] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20240502 07:59:39.599658 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.599725 4905 consensus_queue.cc:1035] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.600515 4074 consensus_queue.cc:1035] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.600596 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 15 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.600700 2877 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.601764 2877 raft_consensus.cc:2454] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
I20240502 07:59:39.601934 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.602031 2878 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.603179 3007 raft_consensus.cc:1261] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 07:59:39.603524 4847 consensus_queue.cc:1035] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.603660 2878 raft_consensus.cc:2396] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 3 because replica has last-logged OpId of term: 2 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 07:59:39.605633 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.605753 2613 raft_consensus.cc:482] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.605819 2613 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.606257 2551 leader_election.cc:302] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:39.606412 4847 raft_consensus.cc:2781] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.606478 4847 raft_consensus.cc:686] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.606622 4847 consensus_queue.cc:237] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.15, Last appended by leader: 15, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.606854 2613 raft_consensus.cc:504] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.607209 2613 leader_election.cc:288] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
W20240502 07:59:39.608011 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45948: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.608800 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.608894 2613 raft_consensus.cc:482] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.608882 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.608942 2613 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.609004 2877 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.609583 3008 raft_consensus.cc:2396] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 3 because replica has last-logged OpId of term: 2 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 07:59:39.609872 2613 raft_consensus.cc:504] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.609974 2877 raft_consensus.cc:2454] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
I20240502 07:59:39.610042 2550 leader_election.cc:302] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b; no voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:39.610219 2613 leader_election.cc:288] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 2 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:39.610219 4905 raft_consensus.cc:2726] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20240502 07:59:39.611055 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.611155 3008 raft_consensus.cc:3032] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Stepping down as leader of term 1
I20240502 07:59:39.611189 3008 raft_consensus.cc:729] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.611305 3008 consensus_queue.cc:260] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.611445 3008 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 1 FOLLOWER]: Advancing to term 2
I20240502 07:59:39.612223 2551 leader_election.cc:302] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:39.612365 3008 raft_consensus.cc:2454] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 2.
I20240502 07:59:39.612401 4905 raft_consensus.cc:2781] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.612490 4905 raft_consensus.cc:686] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.612619 4905 consensus_queue.cc:237] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.6, Last appended by leader: 6, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.612722 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.612803 2613 raft_consensus.cc:482] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.612848 2613 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.613726 2613 raft_consensus.cc:504] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
W20240502 07:59:39.613938 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33724: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.614032 2613 leader_election.cc:288] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
W20240502 07:59:39.614173 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34756: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.614324 2550 leader_election.cc:302] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.615011 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "811167eb170b49219b6cd22d865af334"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:39.615077 4847 raft_consensus.cc:2781] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Leader election won for term 2
I20240502 07:59:39.615114 2613 raft_consensus.cc:482] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.615157 4847 raft_consensus.cc:686] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 2 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.615164 2613 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.615281 4847 consensus_queue.cc:237] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.615731 2466 catalog_manager.cc:5484] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 2 to 3, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 3 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:39.616016 2613 raft_consensus.cc:504] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.616456 2613 leader_election.cc:288] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 4 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:39.616963 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.617069 2878 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.617988 2878 raft_consensus.cc:2454] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
I20240502 07:59:39.618314 2551 leader_election.cc:302] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:39.618463 4905 raft_consensus.cc:2781] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.618530 4905 raft_consensus.cc:686] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.618650 4905 consensus_queue.cc:237] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.619086 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.619197 2877 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 1 FOLLOWER]: Advancing to term 2
W20240502 07:59:39.621556 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.622841 2877 raft_consensus.cc:2454] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 2.
I20240502 07:59:39.624536 2877 raft_consensus.cc:1261] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 07:59:39.624567 2470 catalog_manager.cc:5484] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 2 to 3, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 3 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.624934 4847 consensus_queue.cc:1035] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.625005 2470 catalog_manager.cc:5484] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 2 to 3, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 3 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:39.625162 2470 catalog_manager.cc:5484] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 1 to 2, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 2 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:39.626752 3009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.626881 3009 raft_consensus.cc:3032] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.626921 3009 raft_consensus.cc:729] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.627049 3009 consensus_queue.cc:260] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.627219 3009 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.628227 3009 raft_consensus.cc:2454] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 3.
I20240502 07:59:39.628533 3007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.628639 3007 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.629573 3007 raft_consensus.cc:2454] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 4.
W20240502 07:59:39.630956 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60744: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.632476 2550 leader_election.cc:302] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.632635 4074 raft_consensus.cc:2781] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:39.632704 4074 raft_consensus.cc:686] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 4 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.632813 4074 consensus_queue.cc:237] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
W20240502 07:59:39.633299 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.635023 2466 catalog_manager.cc:5484] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 3 to 4, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 4 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.635143 2878 raft_consensus.cc:1226] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Rejecting Update request from peer e0a355821af5483f96aec7e128cc18e1 for earlier term 2. Current term is 3. Ops: []
W20240502 07:59:39.637679 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60740: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.639374 15878 test_workload.cc:401] TestWorkload: Skipping table creation because table rebalance_test_table_9 already exists
I20240502 07:59:39.641651 4193 consensus_queue.cc:1046] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: INVALID_TERM, Last received: 2.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 07:59:39.641884 4922 raft_consensus.cc:3032] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.641927 4922 raft_consensus.cc:729] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.642047 4922 consensus_queue.cc:260] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 2.16, Last appended by leader: 16, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.642282 4922 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.643700 2748 pending_rounds.cc:76] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1: Aborting all ops after (but not including) 3
I20240502 07:59:39.643754 2748 pending_rounds.cc:97] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 07:59:39.643802 2748 raft_consensus.cc:2866] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 07:59:39.644759 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 15 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.644887 2748 raft_consensus.cc:2396] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 3 because replica has last-logged OpId of term: 2 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 15.
I20240502 07:59:39.645231 2748 raft_consensus.cc:1261] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20240502 07:59:39.646241 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45948: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.646697 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.646785 2748 raft_consensus.cc:3032] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.646818 2748 raft_consensus.cc:729] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.646934 2748 consensus_queue.cc:260] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.7, Last appended by leader: 7, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.647019 2748 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.648180 2748 raft_consensus.cc:2396] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 3 because replica has last-logged OpId of term: 2 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 07:59:39.650740 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.650859 2748 raft_consensus.cc:3032] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.650900 2748 raft_consensus.cc:729] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.651044 2748 consensus_queue.cc:260] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
W20240502 07:59:39.651155 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33736: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.651209 2748 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.652169 2748 raft_consensus.cc:2454] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 4.
I20240502 07:59:39.652498 2748 raft_consensus.cc:1261] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 07:59:39.652957 2877 raft_consensus.cc:1261] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 07:59:39.653728 2877 raft_consensus.cc:1226] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Rejecting Update request from peer e0a355821af5483f96aec7e128cc18e1 for earlier term 2. Current term is 3. Ops: [2.7-2.7]
I20240502 07:59:39.653868 2613 raft_consensus.cc:1226] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 4 LEADER]: Rejecting Update request from peer e0a355821af5483f96aec7e128cc18e1 for earlier term 3. Current term is 4. Ops: []
I20240502 07:59:39.654191 2616 raft_consensus.cc:1226] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Rejecting Update request from peer e0a355821af5483f96aec7e128cc18e1 for earlier term 2. Current term is 3. Ops: [2.16-2.16]
I20240502 07:59:39.654496 2615 raft_consensus.cc:1226] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Rejecting Update request from peer e0a355821af5483f96aec7e128cc18e1 for earlier term 2. Current term is 3. Ops: [2.7-2.7]
I20240502 07:59:39.655787 3010 raft_consensus.cc:1261] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 07:59:39.657042 3007 raft_consensus.cc:1261] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 07:59:39.657611 4517 heartbeater.cc:503] Master 127.15.129.190:34757 was elected leader, sending a full tablet report...
I20240502 07:59:39.657920 4074 consensus_queue.cc:1035] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.658039 4074 consensus_queue.cc:1225] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Peer e0a355821af5483f96aec7e128cc18e1 log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 07:59:39.658828 4074 consensus_queue.cc:1035] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.658929 4074 consensus_queue.cc:1225] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Peer e0a355821af5483f96aec7e128cc18e1 log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20240502 07:59:39.659143 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.660859 4193 consensus_queue.cc:1035] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.660970 4193 consensus_queue.cc:1225] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Peer f1e12e7015044fbeb23a7cd6f4023bef log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 07:59:39.661274 4193 consensus_queue.cc:1035] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:39.662566 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34742: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.662652 3010 raft_consensus.cc:1261] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 07:59:39.663570 2748 raft_consensus.cc:1261] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20240502 07:59:39.665450 4074 consensus_queue.cc:1035] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.667819 2748 pending_rounds.cc:76] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1: Aborting all ops after (but not including) 3
I20240502 07:59:39.667879 2748 pending_rounds.cc:97] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 07:59:39.667917 2748 raft_consensus.cc:2866] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20240502 07:59:39.670543 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34756: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.671047 2748 pending_rounds.cc:76] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1: Aborting all ops after (but not including) 3
I20240502 07:59:39.671095 2748 pending_rounds.cc:97] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 07:59:39.671123 2748 raft_consensus.cc:2866] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 07:59:39.673806 3007 pending_rounds.cc:76] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef: Aborting all ops after (but not including) 3
I20240502 07:59:39.673875 3007 pending_rounds.cc:97] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 07:59:39.673914 3007 raft_consensus.cc:2866] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 07:59:39.674410 4915 consensus_queue.cc:1035] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20240502 07:59:39.674634 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.677793 4910 consensus_queue.cc:1035] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.677924 4910 consensus_queue.cc:1225] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Peer e0a355821af5483f96aec7e128cc18e1 log is divergent from this leader: its last log entry 2.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 07:59:39.680518 2877 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "512e1322925c415f8891dfe6621160e1"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.680680 2877 raft_consensus.cc:482] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.680755 2877 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.681928 2877 raft_consensus.cc:504] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.682713 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.682847 3005 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.683193 4835 mvcc.cc:205] Tried to move back new op lower bound from 7023152249470443520 to 7023152248688222208. Current Snapshot: MvccSnapshot[applied={T|T < 7023152249470443520 or (T in {7023152249470443520})}]
I20240502 07:59:39.683739 2877 leader_election.cc:288] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.684018 3005 raft_consensus.cc:2396] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 4 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 07:59:39.684604 2877 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.684742 2877 raft_consensus.cc:482] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.684811 2877 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.685640 2877 raft_consensus.cc:504] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.686232 2877 leader_election.cc:288] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.686811 2615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.686936 2615 raft_consensus.cc:3032] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.687006 2615 raft_consensus.cc:729] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
W20240502 07:59:39.687077 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33724: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.687170 2615 consensus_queue.cc:260] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.687336 2615 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.687829 2877 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.687938 2877 raft_consensus.cc:482] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.688004 2877 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.688331 2615 raft_consensus.cc:2454] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 4.
I20240502 07:59:39.688809 2877 raft_consensus.cc:504] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
W20240502 07:59:39.689182 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60744: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.689378 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.689440 2813 leader_election.cc:302] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:39.689482 3010 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.689638 4525 raft_consensus.cc:2781] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:39.689846 4525 raft_consensus.cc:686] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.689988 4525 consensus_queue.cc:237] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.690368 3010 raft_consensus.cc:2454] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 4.
I20240502 07:59:39.691159 2812 leader_election.cc:302] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.691315 4525 raft_consensus.cc:2781] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:39.691398 4525 raft_consensus.cc:686] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
W20240502 07:59:39.691515 2576 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45948: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.691536 4525 consensus_queue.cc:237] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.7, Last appended by leader: 7, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.692371 3009 raft_consensus.cc:1226] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Rejecting Update request from peer c93a6d22180a41a686c4556afd6a3e6b for earlier term 3. Current term is 4. Ops: []
I20240502 07:59:39.692314 2475 catalog_manager.cc:5484] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 3 to 4, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 4 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
W20240502 07:59:39.692852 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45984: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.693634 4905 consensus_queue.cc:1046] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: INVALID_TERM, Last received: 3.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:39.693794 4905 raft_consensus.cc:3032] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.693840 4905 raft_consensus.cc:729] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.693965 4905 consensus_queue.cc:260] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 3.7, Last appended by leader: 7, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.694120 4905 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.696691 2470 catalog_manager.cc:5484] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 3 to 4, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 4 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.697597 2615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.698623 2615 raft_consensus.cc:2454] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 4.
I20240502 07:59:39.699250 2877 leader_election.cc:288] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.699676 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.699797 2745 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.700742 2745 raft_consensus.cc:2454] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 4.
I20240502 07:59:39.701038 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.701203 2748 raft_consensus.cc:3032] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.701309 2748 raft_consensus.cc:729] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.701576 2748 consensus_queue.cc:260] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
W20240502 07:59:39.702250 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45978: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.702739 2748 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.703534 2877 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "18912e0059054584a990be7ccb4a5771"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.703662 2877 raft_consensus.cc:482] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.703728 2877 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
W20240502 07:59:39.703814 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60792: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.703953 2748 raft_consensus.cc:2396] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 07:59:39.704339 2815 leader_election.cc:302] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:39.704528 4988 raft_consensus.cc:2726] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 07:59:39.705197 2877 raft_consensus.cc:504] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.705935 2615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.706063 2615 raft_consensus.cc:3032] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.706102 2615 raft_consensus.cc:729] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.706254 2615 consensus_queue.cc:260] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.706429 2615 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
W20240502 07:59:39.706475 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.707007 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 3 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.707134 3008 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.707420 2615 raft_consensus.cc:2454] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 3.
I20240502 07:59:39.707677 2813 leader_election.cc:302] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:39.707873 2877 leader_election.cc:288] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 3 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.708009 4988 raft_consensus.cc:2781] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.708134 4988 raft_consensus.cc:686] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 3 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.708309 4988 consensus_queue.cc:237] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.708954 3008 raft_consensus.cc:2454] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 3.
I20240502 07:59:39.710441 2876 raft_consensus.cc:1261] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20240502 07:59:39.710522 2748 pending_rounds.cc:76] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1: Aborting all ops after (but not including) 3
I20240502 07:59:39.710587 2748 pending_rounds.cc:97] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 07:59:39.710644 2748 raft_consensus.cc:2866] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20240502 07:59:39.712788 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33736: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.713766 2876 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.713874 2876 raft_consensus.cc:482] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.713929 2876 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
W20240502 07:59:39.713912 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60740: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.714514 2470 catalog_manager.cc:5484] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 2 to 3, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 3 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.714954 2876 raft_consensus.cc:504] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.715288 2876 leader_election.cc:288] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 3 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:39.716351 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.716521 2745 raft_consensus.cc:3032] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.716528 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 3 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.716655 3008 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.716611 2745 raft_consensus.cc:729] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.716900 2745 consensus_queue.cc:260] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.717355 2745 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.717721 3008 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 3.
I20240502 07:59:39.718155 2615 raft_consensus.cc:1261] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 3 index: 7. Preceding OpId from leader: term: 4 index: 9. (index mismatch)
I20240502 07:59:39.718564 4988 consensus_queue.cc:1035] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:39.719498 2745 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 3.
I20240502 07:59:39.719805 2815 leader_election.cc:302] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:39.719962 4988 raft_consensus.cc:2781] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.720032 4988 raft_consensus.cc:686] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 3 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.720054 3009 raft_consensus.cc:1261] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 3 index: 7. Preceding OpId from leader: term: 4 index: 9. (index mismatch)
I20240502 07:59:39.720149 4988 consensus_queue.cc:237] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.720746 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.720839 2878 raft_consensus.cc:482] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.720887 2878 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.720914 4988 consensus_queue.cc:1035] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:39.721910 2878 raft_consensus.cc:504] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.722260 2878 leader_election.cc:288] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.722266 4193 consensus_queue.cc:1035] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.722695 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.722831 2616 raft_consensus.cc:3032] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.722877 2616 raft_consensus.cc:729] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.723003 2616 consensus_queue.cc:260] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.723163 2616 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.724197 2616 raft_consensus.cc:2396] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 4 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
W20240502 07:59:39.724499 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.725083 4812 mvcc.cc:205] Tried to move back new op lower bound from 7023152249470443520 to 7023152248688222208. Current Snapshot: MvccSnapshot[applied={T|T < 7023152249470443520}]
I20240502 07:59:39.726966 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.727073 2748 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.727049 2475 catalog_manager.cc:5484] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 2 to 3, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 3 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:39.727653 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.727766 2878 raft_consensus.cc:482] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.727835 2878 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.728041 2748 raft_consensus.cc:2396] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 4 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 07:59:39.728314 2815 leader_election.cc:302] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:39.728520 4525 raft_consensus.cc:2726] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 07:59:39.728818 2878 raft_consensus.cc:504] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.729446 4806 mvcc.cc:205] Tried to move back new op lower bound from 7023152249720205312 to 7023152249617866752. Current Snapshot: MvccSnapshot[applied={T|T < 7023152249539600384}]
I20240502 07:59:39.730043 2878 leader_election.cc:288] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.730069 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 1 index: 14 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.730162 2616 raft_consensus.cc:3032] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.730201 2616 raft_consensus.cc:729] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.730340 2616 consensus_queue.cc:260] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 3.15, Last appended by leader: 15, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.730506 2616 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.731477 2616 raft_consensus.cc:2396] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 4 because replica has last-logged OpId of term: 3 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 14.
I20240502 07:59:39.732146 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.732261 2878 raft_consensus.cc:482] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.732311 2878 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.733171 2878 raft_consensus.cc:504] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.733397 2744 raft_consensus.cc:1261] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20240502 07:59:39.733786 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.733789 4988 consensus_queue.cc:1035] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:39.733880 2613 raft_consensus.cc:3032] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.733919 2613 raft_consensus.cc:729] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.734000 2878 leader_election.cc:288] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.734041 2613 consensus_queue.cc:260] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.7, Last appended by leader: 7, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.734200 2613 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.734246 2744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.734365 2744 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.734599 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.734684 2878 raft_consensus.cc:482] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.734728 2878 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.735121 2613 raft_consensus.cc:2396] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 4 because replica has last-logged OpId of term: 3 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 07:59:39.735240 2744 raft_consensus.cc:2454] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 4.
I20240502 07:59:39.735564 2878 raft_consensus.cc:504] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.735591 2815 leader_election.cc:302] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:39.735805 2616 raft_consensus.cc:1261] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20240502 07:59:39.735919 4988 raft_consensus.cc:2781] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:39.736006 4988 raft_consensus.cc:686] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.736012 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 1 index: 14 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.736097 2748 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.736122 4988 consensus_queue.cc:237] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.736454 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.736547 3008 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.736675 4988 consensus_queue.cc:1035] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:39.737100 2748 raft_consensus.cc:2454] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 4.
I20240502 07:59:39.737458 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.737457 2815 leader_election.cc:302] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:39.737542 2748 raft_consensus.cc:3032] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.737591 2748 raft_consensus.cc:729] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.737673 4525 raft_consensus.cc:2781] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:39.737725 3008 raft_consensus.cc:2454] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 3.
I20240502 07:59:39.737762 2748 consensus_queue.cc:260] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.737766 4525 raft_consensus.cc:686] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.737900 2748 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.737916 4525 consensus_queue.cc:237] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 1.14, Last appended by leader: 14, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.738796 2812 leader_election.cc:302] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.739187 4525 raft_consensus.cc:2781] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.739257 4525 raft_consensus.cc:686] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 3 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.739387 4525 consensus_queue.cc:237] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.739362 2475 catalog_manager.cc:5484] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 3 to 4, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 4 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
W20240502 07:59:39.740128 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34756: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.735904 2878 leader_election.cc:288] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 3 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
W20240502 07:59:39.744166 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34742: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
W20240502 07:59:39.744174 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45948: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.747222 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.747341 2878 raft_consensus.cc:482] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.747395 2878 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.747995 2748 raft_consensus.cc:2396] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 07:59:39.749119 2878 raft_consensus.cc:504] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.749536 2878 leader_election.cc:288] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:39.749833 2877 raft_consensus.cc:1226] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Rejecting Update request from peer e0a355821af5483f96aec7e128cc18e1 for earlier term 3. Current term is 4. Ops: [3.4-3.5]
I20240502 07:59:39.750247 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.750371 3010 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.751400 3010 raft_consensus.cc:2396] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 07:59:39.748008 2470 catalog_manager.cc:5484] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 3 to 4, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 4 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:39.751938 2470 catalog_manager.cc:5484] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 2 to 3, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 3 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.752323 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.752446 2878 raft_consensus.cc:482] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.752517 2878 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.754372 2878 raft_consensus.cc:504] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.754755 2878 leader_election.cc:288] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.755054 2744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 2 index: 16 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.755183 2744 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.756155 2744 raft_consensus.cc:2454] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 4.
I20240502 07:59:39.756402 2815 leader_election.cc:302] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:39.756589 4525 raft_consensus.cc:2781] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:39.756696 4525 raft_consensus.cc:686] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.756736 3009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 2 index: 16 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.756837 3009 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.756851 4525 consensus_queue.cc:237] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 2.16, Last appended by leader: 16, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.757773 3009 raft_consensus.cc:2454] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 4.
I20240502 07:59:39.759140 2876 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.759253 2876 raft_consensus.cc:482] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.754701 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.759317 2876 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.759389 2745 raft_consensus.cc:3032] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.759441 2745 raft_consensus.cc:729] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.759613 2745 consensus_queue.cc:260] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.759765 2745 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.759711 2470 catalog_manager.cc:5484] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 2 to 4, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 4 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:39.760274 2876 raft_consensus.cc:504] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.760569 2876 leader_election.cc:288] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:39.760876 2615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.760993 2615 raft_consensus.cc:3032] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.761034 2615 raft_consensus.cc:729] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.761066 2745 raft_consensus.cc:2396] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20240502 07:59:39.761152 2615 consensus_queue.cc:260] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.761286 2615 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.761372 2815 leader_election.cc:302] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:39.762224 4525 raft_consensus.cc:2726] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20240502 07:59:39.762612 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45984: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.762929 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.763025 3010 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.763253 2615 raft_consensus.cc:2396] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 07:59:39.763958 3010 raft_consensus.cc:2454] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 4.
I20240502 07:59:39.764355 2812 leader_election.cc:302] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters: c93a6d22180a41a686c4556afd6a3e6b
W20240502 07:59:39.764842 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.765615 2877 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.765745 2877 raft_consensus.cc:482] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.765820 2877 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.766837 4525 raft_consensus.cc:2781] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:39.766932 4525 raft_consensus.cc:686] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.767374 4525 consensus_queue.cc:237] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
W20240502 07:59:39.769948 2707 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60744: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
W20240502 07:59:39.770705 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60754: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
W20240502 07:59:39.772935 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33758: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
W20240502 07:59:39.773435 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45992: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.774102 2877 raft_consensus.cc:504] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.774328 2876 raft_consensus.cc:1226] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Rejecting Update request from peer e0a355821af5483f96aec7e128cc18e1 for earlier term 3. Current term is 4. Ops: [3.9-3.9]
I20240502 07:59:39.775017 4193 consensus_queue.cc:1046] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: INVALID_TERM, Last received: 3.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:39.775315 4922 raft_consensus.cc:3032] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.775370 4922 raft_consensus.cc:729] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.775502 4922 consensus_queue.cc:260] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 3.11, Last appended by leader: 11, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.775671 2475 catalog_manager.cc:5484] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 3 to 4, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 4 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:39.775787 4922 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.777074 2877 leader_election.cc:288] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
W20240502 07:59:39.777882 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33752: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.777906 3009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.778506 3009 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.779335 2744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.779491 2744 raft_consensus.cc:2396] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 4 because replica has last-logged OpId of term: 3 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:39.781939 2744 raft_consensus.cc:1261] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 2 index: 14. Preceding OpId from leader: term: 4 index: 18. (index mismatch)
I20240502 07:59:39.782272 4525 consensus_queue.cc:1035] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 07:59:39.782354 2615 raft_consensus.cc:1261] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 3 index: 8. (index mismatch)
I20240502 07:59:39.781790 3008 raft_consensus.cc:1261] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 3 index: 8. (index mismatch)
I20240502 07:59:39.782828 2877 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d7b590b4226443d4899f3005840fd736"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.781036 3009 raft_consensus.cc:2396] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 4 because replica has last-logged OpId of term: 3 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:39.782948 4525 consensus_queue.cc:1035] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:39.783007 2877 raft_consensus.cc:482] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.783141 2812 leader_election.cc:302] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:39.781895 3010 raft_consensus.cc:1261] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 2 index: 16. Preceding OpId from leader: term: 4 index: 18. (index mismatch)
W20240502 07:59:39.785017 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.785214 4990 raft_consensus.cc:2726] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 07:59:39.785346 4990 consensus_queue.cc:1035] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20240502 07:59:39.785542 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34830: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.786007 4525 consensus_queue.cc:1035] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 07:59:39.783142 2877 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
W20240502 07:59:39.787959 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34814: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.788506 2877 raft_consensus.cc:504] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.788854 2877 leader_election.cc:288] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.789470 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.789577 3008 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.789851 2744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.789963 2744 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.790902 3008 raft_consensus.cc:2454] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 4.
I20240502 07:59:39.791801 2812 leader_election.cc:302] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.791945 4988 raft_consensus.cc:2781] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:39.792042 4988 raft_consensus.cc:686] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.792184 4988 consensus_queue.cc:237] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.793042 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c67533803b9c496295f4f57e77a51374"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.793143 2878 raft_consensus.cc:482] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.793192 2878 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.794224 2878 raft_consensus.cc:504] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.794554 2878 leader_election.cc:288] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 3 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:39.794940 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.795048 3008 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.795951 3008 raft_consensus.cc:2454] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 3.
I20240502 07:59:39.796670 2744 raft_consensus.cc:2454] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 4.
I20240502 07:59:39.796998 2812 leader_election.cc:302] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.797165 4991 raft_consensus.cc:2781] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election won for term 3
W20240502 07:59:39.797413 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60754: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.797446 4991 raft_consensus.cc:686] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 3 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
W20240502 07:59:39.797540 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33736: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.797585 4991 consensus_queue.cc:237] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.797857 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.797953 2748 raft_consensus.cc:3032] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.797991 2748 raft_consensus.cc:729] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:39.798188 2748 consensus_queue.cc:260] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.798867 2748 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.799840 2748 raft_consensus.cc:2454] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 3.
I20240502 07:59:39.799887 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.800012 2878 raft_consensus.cc:482] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.800063 2878 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.800953 2878 raft_consensus.cc:504] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.801236 2878 leader_election.cc:288] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 3 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:39.801620 3009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 3 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.802021 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 3 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.802145 2613 raft_consensus.cc:3032] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.802193 2613 raft_consensus.cc:729] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
W20240502 07:59:39.802361 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60784: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.802345 2613 consensus_queue.cc:260] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.10, Last appended by leader: 10, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.802503 2613 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.803522 2613 raft_consensus.cc:2396] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 3 because replica has last-logged OpId of term: 2 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 9.
W20240502 07:59:39.807268 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45948: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.807971 2470 catalog_manager.cc:5484] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 2 to 4, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 4 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.808305 2470 catalog_manager.cc:5484] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 2 to 3, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 3 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
W20240502 07:59:39.806828 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33752: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.808816 3009 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.810256 3009 raft_consensus.cc:2454] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 3.
I20240502 07:59:39.810546 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.810663 2878 raft_consensus.cc:482] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.810724 2878 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.811218 2812 leader_election.cc:302] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:39.811668 2878 raft_consensus.cc:504] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.811847 4988 raft_consensus.cc:2781] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.811918 4988 raft_consensus.cc:686] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 3 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.812055 4988 consensus_queue.cc:237] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.812358 2878 leader_election.cc:288] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.813185 2615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.813422 2615 raft_consensus.cc:3032] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.813524 2615 raft_consensus.cc:729] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.813752 2615 consensus_queue.cc:260] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 3.12, Last appended by leader: 12, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
W20240502 07:59:39.813864 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45992: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
W20240502 07:59:39.816138 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34830: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.816684 2615 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.816740 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.816860 3008 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.817811 2615 raft_consensus.cc:2396] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 4 because replica has last-logged OpId of term: 3 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 11.
I20240502 07:59:39.817884 3008 raft_consensus.cc:2454] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 4.
I20240502 07:59:39.819257 2812 leader_election.cc:302] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:39.819475 4525 raft_consensus.cc:2781] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:39.819550 4525 raft_consensus.cc:686] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.819666 4525 consensus_queue.cc:237] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.820492 3008 raft_consensus.cc:1261] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20240502 07:59:39.820636 2748 raft_consensus.cc:1261] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20240502 07:59:39.820859 4074 consensus_queue.cc:1035] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:39.821084 4905 consensus_queue.cc:1035] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
W20240502 07:59:39.822551 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34814: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.825903 2466 catalog_manager.cc:5484] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 2 to 3, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 3 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:39.827600 2748 raft_consensus.cc:1261] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 1 index: 13. Preceding OpId from leader: term: 4 index: 16. (index mismatch)
I20240502 07:59:39.827989 4988 consensus_queue.cc:1035] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 07:59:39.828116 2615 raft_consensus.cc:1261] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 3 index: 15. Preceding OpId from leader: term: 4 index: 16. (index mismatch)
W20240502 07:59:39.829088 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60754: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.829900 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.830021 2878 raft_consensus.cc:482] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.830085 2878 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
W20240502 07:59:39.830068 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60784: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
W20240502 07:59:39.830907 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.831362 4988 consensus_queue.cc:1035] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 07:59:39.831459 4988 consensus_queue.cc:1225] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Peer c93a6d22180a41a686c4556afd6a3e6b log is divergent from this leader: its last log entry 3.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20240502 07:59:39.831895 2878 raft_consensus.cc:504] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.833298 2878 leader_election.cc:288] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.833348 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.833439 2747 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.834437 2747 raft_consensus.cc:2396] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 07:59:39.835819 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.835930 2878 raft_consensus.cc:482] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.835991 2878 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.836201 2748 raft_consensus.cc:1261] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 4 index: 9. (index mismatch)
I20240502 07:59:39.836567 4991 consensus_queue.cc:1035] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:39.836952 2878 raft_consensus.cc:504] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
W20240502 07:59:39.839421 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33752: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.839545 2878 leader_election.cc:288] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
W20240502 07:59:39.839962 2576 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45992: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.841158 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.841264 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.841274 2616 raft_consensus.cc:3032] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.841392 2616 raft_consensus.cc:729] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.841408 2613 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.841564 2616 consensus_queue.cc:260] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.841758 2616 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.842370 2613 raft_consensus.cc:2454] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 4.
I20240502 07:59:39.842957 3010 raft_consensus.cc:1261] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 2 index: 7. Preceding OpId from leader: term: 4 index: 9. (index mismatch)
I20240502 07:59:39.842828 2616 raft_consensus.cc:2396] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 07:59:39.843739 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.843840 3010 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.843832 2813 leader_election.cc:302] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:39.844002 4991 raft_consensus.cc:2781] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:39.844024 2813 leader_election.cc:302] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:39.844127 4991 raft_consensus.cc:686] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.844306 4991 consensus_queue.cc:237] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.844316 4990 raft_consensus.cc:2726] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 07:59:39.844846 3010 raft_consensus.cc:2454] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 4.
I20240502 07:59:39.845999 4990 consensus_queue.cc:1035] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20240502 07:59:39.847364 2475 catalog_manager.cc:5484] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 3 to 4, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 4 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:39.847601 2475 catalog_manager.cc:5484] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 2 to 4, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 4 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
W20240502 07:59:39.849624 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60744: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.849807 2615 pending_rounds.cc:76] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b: Aborting all ops after (but not including) 14
I20240502 07:59:39.849874 2615 pending_rounds.cc:97] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b: Aborting uncommitted NO_OP operation due to leader change: 3.15
I20240502 07:59:39.849941 2615 raft_consensus.cc:2866] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 07:59:39.850981 4650 heartbeater.cc:503] Master 127.15.129.190:34757 was elected leader, sending a full tablet report...
I20240502 07:59:39.851678 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.851796 2878 raft_consensus.cc:482] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.851847 2878 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.852836 2878 raft_consensus.cc:504] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
W20240502 07:59:39.854024 2834 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34830: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.855149 2878 leader_election.cc:288] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
W20240502 07:59:39.855392 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.856187 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 2 index: 15 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.856297 2614 raft_consensus.cc:3032] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.856334 2614 raft_consensus.cc:729] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.856448 2614 consensus_queue.cc:260] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 3.16, Last appended by leader: 16, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.856587 2614 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.857618 2614 raft_consensus.cc:2396] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 4 because replica has last-logged OpId of term: 3 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 15.
W20240502 07:59:39.858093 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34814: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.855373 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 2 index: 15 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.858518 2748 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.859575 2748 raft_consensus.cc:2396] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 4 because replica has last-logged OpId of term: 2 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 15.
I20240502 07:59:39.860503 2815 leader_election.cc:302] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:39.860816 4990 raft_consensus.cc:2726] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 07:59:39.863873 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.863983 2878 raft_consensus.cc:482] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.864094 2878 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.864898 2614 raft_consensus.cc:1261] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 3 index: 7. Preceding OpId from leader: term: 4 index: 8. (index mismatch)
I20240502 07:59:39.865084 2878 raft_consensus.cc:504] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.865401 4990 consensus_queue.cc:1035] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:39.865487 4990 consensus_queue.cc:1225] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Peer c93a6d22180a41a686c4556afd6a3e6b log is divergent from this leader: its last log entry 3.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20240502 07:59:39.865617 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60754: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.866086 2614 pending_rounds.cc:76] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b: Aborting all ops after (but not including) 6
I20240502 07:59:39.866128 2614 pending_rounds.cc:97] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b: Aborting uncommitted NO_OP operation due to leader change: 3.7
I20240502 07:59:39.866154 2614 raft_consensus.cc:2866] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 07:59:39.866709 2878 leader_election.cc:288] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.868073 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "236a4d787879442aaff6983243f21f85"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.868188 2878 raft_consensus.cc:482] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.868238 2878 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
W20240502 07:59:39.868733 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60784: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.869904 2878 raft_consensus.cc:504] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.870317 2878 leader_election.cc:288] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:39.870760 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 3 index: 20 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:39.870878 3010 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.870867 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.870972 2614 raft_consensus.cc:3032] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.871016 2614 raft_consensus.cc:729] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.871137 2614 consensus_queue.cc:260] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.7, Last appended by leader: 7, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.871263 2614 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.871919 3010 raft_consensus.cc:2454] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 4.
I20240502 07:59:39.872267 2614 raft_consensus.cc:2396] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 4 because replica has last-logged OpId of term: 3 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 07:59:39.872474 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.872586 2747 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.873514 2744 raft_consensus.cc:1261] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 2 index: 6. Preceding OpId from leader: term: 4 index: 8. (index mismatch)
I20240502 07:59:39.873571 2747 raft_consensus.cc:2396] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 4 because replica has last-logged OpId of term: 2 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 07:59:39.875945 2812 leader_election.cc:302] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.876312 4991 raft_consensus.cc:2781] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:39.876399 4991 raft_consensus.cc:686] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.876535 4991 consensus_queue.cc:237] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 3.20, Last appended by leader: 20, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.876915 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:39.877036 2878 raft_consensus.cc:482] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
W20240502 07:59:39.877804 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33752: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.877082 2878 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.878787 2878 raft_consensus.cc:504] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
W20240502 07:59:39.879416 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45948: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.879441 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.879540 2614 raft_consensus.cc:3032] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.879575 2614 raft_consensus.cc:729] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.879698 2614 consensus_queue.cc:260] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.879840 2614 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
W20240502 07:59:39.880123 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45992: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.875020 2815 leader_election.cc:302] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:39.880542 4988 raft_consensus.cc:2726] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 07:59:39.880569 4990 consensus_queue.cc:1035] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:39.880743 2614 raft_consensus.cc:2454] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 4.
I20240502 07:59:39.881597 2878 leader_election.cc:288] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.882917 2470 catalog_manager.cc:5484] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 3 to 4, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 4 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:39.885166 2813 leader_election.cc:302] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:39.885564 4525 raft_consensus.cc:2781] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:39.885644 4525 raft_consensus.cc:686] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.885751 4525 consensus_queue.cc:237] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.887651 2615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 3 index: 20 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.887763 2615 raft_consensus.cc:3032] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.887804 2615 raft_consensus.cc:729] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.887974 2615 consensus_queue.cc:260] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 3.20, Last appended by leader: 20, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.888157 2615 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.889055 2744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.889154 2744 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.890213 2744 raft_consensus.cc:2454] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 4.
I20240502 07:59:39.890627 3009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "512e1322925c415f8891dfe6621160e1"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.890813 3009 raft_consensus.cc:482] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.890935 3009 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
W20240502 07:59:39.892367 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34830: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.894173 3009 raft_consensus.cc:504] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
W20240502 07:59:39.894292 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34814: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.894765 3009 leader_election.cc:288] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.895341 3009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.895488 3009 raft_consensus.cc:482] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.895581 3009 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.895710 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.895844 2878 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.896332 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.896445 2747 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.896975 2878 raft_consensus.cc:2454] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 5.
I20240502 07:59:39.897684 2944 leader_election.cc:302] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.897855 4853 raft_consensus.cc:2781] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:39.897934 4853 raft_consensus.cc:686] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.898058 4853 consensus_queue.cc:237] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.898365 2747 raft_consensus.cc:2396] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20240502 07:59:39.898057 3009 raft_consensus.cc:504] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.900413 2470 catalog_manager.cc:5484] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 3 to 5, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 5 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.900537 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.900700 2878 raft_consensus.cc:3032] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:39.900784 2878 raft_consensus.cc:729] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.900936 2615 raft_consensus.cc:2454] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 4.
I20240502 07:59:39.900944 2878 consensus_queue.cc:260] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 4.10, Last appended by leader: 10, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.901189 2878 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.902290 2878 raft_consensus.cc:2454] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 5.
I20240502 07:59:39.902590 2944 leader_election.cc:302] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.902752 4853 raft_consensus.cc:2781] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:39.902834 4853 raft_consensus.cc:686] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.902952 4853 consensus_queue.cc:237] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 4.10, Last appended by leader: 10, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.903978 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.904093 2614 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.904300 3010 raft_consensus.cc:1261] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20240502 07:59:39.904237 2475 catalog_manager.cc:5484] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 3 to 4, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 4 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.905071 2614 raft_consensus.cc:2454] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 5.
I20240502 07:59:39.904897 2470 catalog_manager.cc:5484] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 4 to 5, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 5 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.905587 3009 leader_election.cc:288] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
W20240502 07:59:39.906383 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.906805 3009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.906963 3009 raft_consensus.cc:482] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.907086 3009 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.908030 3009 raft_consensus.cc:504] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.908836 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 3 index: 14 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.909029 2614 raft_consensus.cc:3032] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:39.909106 2614 raft_consensus.cc:729] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.909293 2614 consensus_queue.cc:260] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 4.15, Last appended by leader: 15, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.909551 2614 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.910589 2614 raft_consensus.cc:2396] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 5 because replica has last-logged OpId of term: 4 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 14.
W20240502 07:59:39.910777 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60754: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
W20240502 07:59:39.911398 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60784: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.911419 3009 leader_election.cc:288] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.912003 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "236a4d787879442aaff6983243f21f85"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.911394 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 3 index: 14 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.912101 3010 raft_consensus.cc:482] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.912103 2747 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.912149 3010 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.913017 3010 raft_consensus.cc:504] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.913041 2747 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 5.
I20240502 07:59:39.913307 3010 leader_election.cc:288] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.913691 2946 leader_election.cc:302] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:39.913857 4853 raft_consensus.cc:2781] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:39.913935 4853 raft_consensus.cc:686] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.914038 4853 consensus_queue.cc:237] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 3.14, Last appended by leader: 14, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.915405 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.915498 3005 raft_consensus.cc:482] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.915540 3005 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.916050 2615 raft_consensus.cc:1261] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20240502 07:59:39.916337 3005 raft_consensus.cc:504] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.916451 4991 consensus_queue.cc:1035] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:39.916569 4991 consensus_queue.cc:1225] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Peer c93a6d22180a41a686c4556afd6a3e6b log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20240502 07:59:39.916615 3005 leader_election.cc:288] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.916895 4991 consensus_queue.cc:1035] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:39.918143 2475 catalog_manager.cc:5484] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 4 to 5, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 5 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.918848 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.918946 3005 raft_consensus.cc:482] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.918994 3005 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.919850 3005 raft_consensus.cc:504] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.920176 3010 raft_consensus.cc:1226] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Rejecting Update request from peer c93a6d22180a41a686c4556afd6a3e6b for earlier term 2. Current term is 3. Ops: []
I20240502 07:59:39.920209 3005 leader_election.cc:288] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 3 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.920585 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 3 index: 20 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.920761 2614 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.921025 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 3 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.921142 2613 raft_consensus.cc:3032] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 2 LEADER]: Stepping down as leader of term 2
I20240502 07:59:39.921195 2613 raft_consensus.cc:729] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.921311 2613 consensus_queue.cc:260] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 2.17, Last appended by leader: 17, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
W20240502 07:59:39.921514 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45912: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.921540 2613 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.922374 2614 raft_consensus.cc:2454] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 5.
I20240502 07:59:39.922518 2613 raft_consensus.cc:2454] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 3.
I20240502 07:59:39.922698 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 3 index: 20 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.922837 2878 raft_consensus.cc:3032] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:39.922909 2878 raft_consensus.cc:729] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.922905 2944 leader_election.cc:302] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.923059 2944 leader_election.cc:302] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.923190 4853 raft_consensus.cc:2781] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:39.923267 4853 raft_consensus.cc:686] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.923370 4853 consensus_queue.cc:237] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 3.20, Last appended by leader: 20, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.923802 3005 raft_consensus.cc:1226] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Rejecting Update request from peer c93a6d22180a41a686c4556afd6a3e6b for earlier term 4. Current term is 5. Ops: []
I20240502 07:59:39.923202 4883 raft_consensus.cc:2781] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Leader election won for term 3
I20240502 07:59:39.924124 4883 raft_consensus.cc:686] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 3 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.924218 4883 consensus_queue.cc:237] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 2.17, Last appended by leader: 17, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.924366 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.924465 3005 raft_consensus.cc:482] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.924513 3005 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.924965 2744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 3 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.925016 4910 consensus_queue.cc:1046] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.925072 2744 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 2 FOLLOWER]: Advancing to term 3
I20240502 07:59:39.925143 4910 raft_consensus.cc:3032] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:39.925179 4910 raft_consensus.cc:729] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.925282 4910 consensus_queue.cc:260] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.925494 4910 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.926031 2475 catalog_manager.cc:5484] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 4 to 5, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 5 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:39.926393 2744 raft_consensus.cc:2454] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 3.
I20240502 07:59:39.926255 2475 catalog_manager.cc:5484] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 2 to 3, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 3 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.926767 2878 consensus_queue.cc:260] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 4.21, Last appended by leader: 21, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.926975 2878 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
W20240502 07:59:39.926976 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33752: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.927274 3005 raft_consensus.cc:504] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.927714 3005 leader_election.cc:288] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.927862 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.927968 2747 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.928182 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.928290 2877 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.928747 2878 raft_consensus.cc:2396] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 5 because replica has last-logged OpId of term: 4 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 20.
I20240502 07:59:39.929124 2877 raft_consensus.cc:2454] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 5.
I20240502 07:59:39.929173 2747 raft_consensus.cc:2396] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 07:59:39.929286 2615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.929486 2615 raft_consensus.cc:2396] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 07:59:39.929647 2944 leader_election.cc:302] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.929874 4883 raft_consensus.cc:2781] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:39.929958 4883 raft_consensus.cc:686] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.930070 4883 consensus_queue.cc:237] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.931329 2946 leader_election.cc:302] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:39.931530 4885 raft_consensus.cc:2726] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20240502 07:59:39.931777 2615 pending_rounds.cc:76] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b: Aborting all ops after (but not including) 5
I20240502 07:59:39.931874 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "18912e0059054584a990be7ccb4a5771"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.931906 2615 pending_rounds.cc:97] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20240502 07:59:39.931978 3005 raft_consensus.cc:482] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.932039 3005 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.932070 2744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.932154 2744 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.932435 2616 raft_consensus.cc:1261] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 2 index: 17. Preceding OpId from leader: term: 3 index: 19. (index mismatch)
W20240502 07:59:39.931924 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45992: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
W20240502 07:59:39.932785 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.932854 2615 raft_consensus.cc:2866] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 07:59:39.932770 4885 consensus_queue.cc:1035] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240502 07:59:39.933265 2744 raft_consensus.cc:2454] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 5.
I20240502 07:59:39.934041 2745 raft_consensus.cc:1226] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Rejecting Update request from peer c93a6d22180a41a686c4556afd6a3e6b for earlier term 2. Current term is 3. Ops: [2.17-2.17]
I20240502 07:59:39.934461 2475 catalog_manager.cc:5484] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 3 to 5, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 5 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:39.933547 3005 raft_consensus.cc:504] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.937908 3005 leader_election.cc:288] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 4 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:39.937727 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.938596 2616 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.940062 2747 raft_consensus.cc:1261] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 2 index: 16. Preceding OpId from leader: term: 3 index: 19. (index mismatch)
I20240502 07:59:39.941203 3005 raft_consensus.cc:1226] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Rejecting Update request from peer 606c439025f24e53a2feebeaf68c1681 for earlier term 3. Current term is 4. Ops: []
I20240502 07:59:39.941380 4885 consensus_queue.cc:1035] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 07:59:39.943467 2616 raft_consensus.cc:2454] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 4.
I20240502 07:59:39.943693 2614 raft_consensus.cc:1226] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Rejecting Update request from peer 606c439025f24e53a2feebeaf68c1681 for earlier term 3. Current term is 4. Ops: [3.13-3.13]
I20240502 07:59:39.944036 2944 leader_election.cc:302] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
W20240502 07:59:39.944415 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34830: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.945091 4993 consensus_queue.cc:1046] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: INVALID_TERM, Last received: 3.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:39.945241 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.945297 4993 raft_consensus.cc:3032] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.945433 4993 raft_consensus.cc:729] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.945573 4993 consensus_queue.cc:260] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 3.13, Last appended by leader: 13, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.945814 4993 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.945840 4853 raft_consensus.cc:2781] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:39.945922 4853 raft_consensus.cc:686] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.946036 4853 consensus_queue.cc:237] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 3.12, Last appended by leader: 12, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:39.946282 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f116334237934021a25a6cacae430034"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.946390 3005 raft_consensus.cc:482] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.946441 3005 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.950624 2877 raft_consensus.cc:2396] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 4 because replica has last-logged OpId of term: 3 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
W20240502 07:59:39.951375 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34814: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.951684 3005 raft_consensus.cc:504] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.952271 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.952404 2614 raft_consensus.cc:3032] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.952456 2614 raft_consensus.cc:729] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:39.952611 2614 consensus_queue.cc:260] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.952806 2614 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.953840 2614 raft_consensus.cc:2454] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 4.
I20240502 07:59:39.954092 2944 leader_election.cc:302] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.955011 4853 raft_consensus.cc:2781] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:39.955093 4853 raft_consensus.cc:686] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.954983 2470 catalog_manager.cc:5484] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 3 to 4, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 4 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:39.955199 4853 consensus_queue.cc:237] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.955575 3005 leader_election.cc:288] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 4 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.957172 2744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.957612 2744 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.957954 3008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.958076 3008 raft_consensus.cc:482] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.958142 3008 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.958911 2744 raft_consensus.cc:2454] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 4.
W20240502 07:59:39.961486 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45948: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
W20240502 07:59:39.967551 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60784: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.968554 2466 catalog_manager.cc:5484] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 3 to 4, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 4 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:39.969077 3008 raft_consensus.cc:504] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.970602 2877 raft_consensus.cc:1261] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20240502 07:59:39.973232 4883 consensus_queue.cc:1035] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:39.974751 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 4 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.977582 2748 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.978895 2748 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 4.
I20240502 07:59:39.979043 2747 raft_consensus.cc:1261] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20240502 07:59:39.981635 3008 leader_election.cc:288] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 4 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.982237 3009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.982375 3009 raft_consensus.cc:482] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.982434 3009 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.983449 3009 raft_consensus.cc:504] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.983523 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 4 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.983647 2878 raft_consensus.cc:3032] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:39.983686 2878 raft_consensus.cc:729] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.983831 3009 leader_election.cc:288] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.983806 2878 consensus_queue.cc:260] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 3.12, Last appended by leader: 12, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.983939 2878 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.984983 2878 raft_consensus.cc:2396] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 4 because replica has last-logged OpId of term: 3 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 11.
I20240502 07:59:39.985456 2946 leader_election.cc:302] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.985457 4885 consensus_queue.cc:1035] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:39.985673 4855 raft_consensus.cc:2781] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:39.985770 4855 raft_consensus.cc:686] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:39.985886 4855 consensus_queue.cc:237] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 2.11, Last appended by leader: 11, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.985657 3009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d7b590b4226443d4899f3005840fd736"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.986711 3009 raft_consensus.cc:482] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.986773 3009 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.987696 3009 raft_consensus.cc:504] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.987995 3009 leader_election.cc:288] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:39.988528 2744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.988600 3009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c67533803b9c496295f4f57e77a51374"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.988718 3009 raft_consensus.cc:482] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.988754 2744 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.988816 3009 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.989770 3009 raft_consensus.cc:504] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
W20240502 07:59:39.991796 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45992: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:39.992494 3009 leader_election.cc:288] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 4 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.993011 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.993010 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.993132 2878 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.993132 3010 raft_consensus.cc:482] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.993185 3010 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:39.994143 2748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:39.994143 3010 raft_consensus.cc:504] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:39.994246 2748 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.994257 2878 raft_consensus.cc:2454] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 5.
I20240502 07:59:39.994482 3010 leader_election.cc:288] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 4 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
W20240502 07:59:39.994695 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60754: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.995056 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.995157 3010 raft_consensus.cc:482] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.995209 3010 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.996063 3010 raft_consensus.cc:504] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:39.996294 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.996379 3010 leader_election.cc:288] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:39.996390 2878 raft_consensus.cc:3032] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:39.996426 2878 raft_consensus.cc:729] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:39.996554 2878 consensus_queue.cc:260] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 4.11, Last appended by leader: 11, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:39.996649 2878 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:39.997509 2878 raft_consensus.cc:2396] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 5 because replica has last-logged OpId of term: 4 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 10.
W20240502 07:59:39.998209 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:39.998432 2748 raft_consensus.cc:2454] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 5.
I20240502 07:59:39.998608 2745 raft_consensus.cc:1226] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Rejecting Update request from peer 606c439025f24e53a2feebeaf68c1681 for earlier term 4. Current term is 5. Ops: [4.11-4.11]
W20240502 07:59:39.998823 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33752: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:39.999106 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 4 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:39.999500 2946 leader_election.cc:302] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:39.999756 3009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:39.999848 3009 raft_consensus.cc:482] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:39.999966 4855 raft_consensus.cc:2781] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.000056 4855 raft_consensus.cc:686] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.000174 4855 consensus_queue.cc:237] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 4.10, Last appended by leader: 10, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.000692 3010 raft_consensus.cc:1226] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Rejecting Update request from peer 606c439025f24e53a2feebeaf68c1681 for earlier term 4. Current term is 5. Ops: [4.11-4.11]
I20240502 07:59:40.001214 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 4 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:39.999963 3009 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.001391 2878 raft_consensus.cc:3032] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:40.001514 2878 raft_consensus.cc:729] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.001705 2878 consensus_queue.cc:260] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 3.10, Last appended by leader: 10, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.002133 2878 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:40.003180 3009 raft_consensus.cc:504] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.003264 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.003346 2877 raft_consensus.cc:3032] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:40.003381 2877 raft_consensus.cc:729] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.003508 3009 leader_election.cc:288] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.003499 2877 consensus_queue.cc:260] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.003634 2877 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:40.003749 3010 raft_consensus.cc:1261] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 8. (index mismatch)
I20240502 07:59:40.003854 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.004187 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.004279 3010 raft_consensus.cc:482] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.004340 3010 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.004309 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 4 index: 25 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.004407 2745 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.004591 2877 raft_consensus.cc:2396] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 07:59:40.005174 3010 raft_consensus.cc:504] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.005277 2745 raft_consensus.cc:2454] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 5.
I20240502 07:59:40.005513 2878 raft_consensus.cc:2396] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 4 because replica has last-logged OpId of term: 3 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 9.
I20240502 07:59:40.005553 3010 leader_election.cc:288] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.000721 2614 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:40.005566 2946 leader_election.cc:302] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.005903 2745 raft_consensus.cc:1261] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 8. (index mismatch)
I20240502 07:59:40.006014 4885 raft_consensus.cc:2781] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.006099 4885 raft_consensus.cc:686] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.006217 4885 consensus_queue.cc:237] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 4.25, Last appended by leader: 25, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.007110 2614 raft_consensus.cc:2396] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 4 because replica has last-logged OpId of term: 2 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 9.
I20240502 07:59:40.007282 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.007400 3010 raft_consensus.cc:482] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.007463 3010 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.001016 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.007541 2616 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.008303 3010 raft_consensus.cc:504] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.004148 4074 consensus_queue.cc:1035] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20240502 07:59:40.008620 3010 leader_election.cc:288] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.008881 4910 consensus_queue.cc:1035] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:40.009392 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.009485 2745 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.009614 2616 raft_consensus.cc:2396] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 5 because replica has last-logged OpId of term: 3 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 11.
I20240502 07:59:40.010380 2745 raft_consensus.cc:2454] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 5.
I20240502 07:59:40.010375 2747 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:40.011001 3008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.011111 3008 raft_consensus.cc:482] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.011179 3008 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:40.011873 2744 raft_consensus.cc:2454] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 5.
I20240502 07:59:40.011997 2944 leader_election.cc:302] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.012423 4855 raft_consensus.cc:2781] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.012498 4855 raft_consensus.cc:686] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.012617 4855 consensus_queue.cc:237] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 3.11, Last appended by leader: 11, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.013092 2946 leader_election.cc:302] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.013387 4885 raft_consensus.cc:2781] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.013476 4885 raft_consensus.cc:686] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.013612 4885 consensus_queue.cc:237] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
W20240502 07:59:40.013949 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34814: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.014415 2744 raft_consensus.cc:1226] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Rejecting Update request from peer 606c439025f24e53a2feebeaf68c1681 for earlier term 4. Current term is 5. Ops: [4.26-4.26]
I20240502 07:59:40.014530 2944 leader_election.cc:302] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.014782 4991 consensus_queue.cc:1046] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: INVALID_TERM, Last received: 4.25, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20240502 07:59:40.015024 4993 raft_consensus.cc:3032] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:40.015075 4993 raft_consensus.cc:729] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.015185 4993 consensus_queue.cc:260] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 4.26, Last appended by leader: 26, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.015524 4993 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.017095 4885 raft_consensus.cc:2726] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 07:59:40.017273 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.017398 2616 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.018425 2616 raft_consensus.cc:2396] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 5 because replica has last-logged OpId of term: 4 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 8.
I20240502 07:59:40.018963 2876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.019071 2876 raft_consensus.cc:3032] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:40.019114 2876 raft_consensus.cc:729] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.019275 2747 raft_consensus.cc:2454] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 4.
I20240502 07:59:40.019268 2876 consensus_queue.cc:260] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 4.9, Last appended by leader: 9, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.019415 2876 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.019541 2946 leader_election.cc:302] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681
I20240502 07:59:40.019747 4855 raft_consensus.cc:2781] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:40.019812 4855 raft_consensus.cc:686] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.019927 4855 consensus_queue.cc:237] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.020377 2876 raft_consensus.cc:2396] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 5 because replica has last-logged OpId of term: 4 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 8.
I20240502 07:59:40.020659 3008 raft_consensus.cc:504] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.020918 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 4 index: 25 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.021046 2877 raft_consensus.cc:2396] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 5 because replica has last-logged OpId of term: 4 index: 26, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 25.
I20240502 07:59:40.021287 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.021411 2747 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:40.021932 3008 leader_election.cc:288] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 4 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.022171 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.021272 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.022464 2878 raft_consensus.cc:3032] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:40.022598 2614 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.022604 3009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.022732 3009 raft_consensus.cc:482] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.022792 3009 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.022903 2878 raft_consensus.cc:729] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.023098 2747 raft_consensus.cc:2396] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 07:59:40.023710 3009 raft_consensus.cc:504] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.023995 3009 leader_election.cc:288] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.024112 2944 leader_election.cc:302] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.024264 4885 raft_consensus.cc:2726] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20240502 07:59:40.024626 3009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "811167eb170b49219b6cd22d865af334"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.024726 3009 raft_consensus.cc:482] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.024782 3009 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.025657 3009 raft_consensus.cc:504] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.025012 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.025971 2616 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.026000 3009 leader_election.cc:288] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.026791 2614 raft_consensus.cc:2454] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 5.
I20240502 07:59:40.026938 3009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.025367 2876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.025467 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.027053 3009 raft_consensus.cc:482] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.026968 2616 raft_consensus.cc:2454] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 5.
I20240502 07:59:40.027067 2876 raft_consensus.cc:3032] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:40.027084 2879 raft_consensus.cc:3032] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:40.027117 3009 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:40.027114 2876 raft_consensus.cc:729] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.027132 2879 raft_consensus.cc:729] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.027261 2879 consensus_queue.cc:260] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.8, Last appended by leader: 8, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.027259 2876 consensus_queue.cc:260] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.027431 2879 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.027554 2876 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:40.027797 2744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.028441 2876 raft_consensus.cc:2396] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 07:59:40.028529 2879 raft_consensus.cc:2396] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 5 because replica has last-logged OpId of term: 4 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 7.
I20240502 07:59:40.027887 2744 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.029842 3009 raft_consensus.cc:504] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.030139 3009 leader_election.cc:288] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 4 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.030300 2745 raft_consensus.cc:1261] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 3 index: 8. Preceding OpId from leader: term: 5 index: 10. (index mismatch)
I20240502 07:59:40.030288 2878 consensus_queue.cc:260] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 4.12, Last appended by leader: 12, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.030705 2878 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.031081 2613 raft_consensus.cc:1261] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 10. (index mismatch)
I20240502 07:59:40.031297 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.031412 2745 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:40.031603 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.031692 2878 raft_consensus.cc:2396] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 5 because replica has last-logged OpId of term: 4 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 11.
I20240502 07:59:40.031692 2613 raft_consensus.cc:3032] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:40.031728 2613 raft_consensus.cc:729] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.031843 2613 consensus_queue.cc:260] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.031977 2613 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:40.032168 3008 raft_consensus.cc:1226] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Rejecting Update request from peer 606c439025f24e53a2feebeaf68c1681 for earlier term 4. Current term is 5. Ops: [4.26-4.26]
I20240502 07:59:40.032558 2744 raft_consensus.cc:2454] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 5.
I20240502 07:59:40.032814 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.032824 2946 leader_election.cc:302] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.032910 2616 raft_consensus.cc:3032] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:40.032881 4855 consensus_queue.cc:1035] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20240502 07:59:40.032934 2613 raft_consensus.cc:2454] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 4.
I20240502 07:59:40.033149 4855 raft_consensus.cc:2781] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.033229 4855 raft_consensus.cc:686] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.032945 2616 raft_consensus.cc:729] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.033437 4855 consensus_queue.cc:237] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 4.8, Last appended by leader: 8, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
W20240502 07:59:40.033502 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60784: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.033506 2616 consensus_queue.cc:260] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 4.9, Last appended by leader: 9, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.033620 2616 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.034313 2745 raft_consensus.cc:2454] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 4.
I20240502 07:59:40.034381 3005 raft_consensus.cc:1226] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Rejecting Update request from peer 606c439025f24e53a2feebeaf68c1681 for earlier term 4. Current term is 5. Ops: [4.9-4.9]
I20240502 07:59:40.034572 2616 raft_consensus.cc:2396] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 5 because replica has last-logged OpId of term: 4 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 8.
I20240502 07:59:40.034689 2946 leader_election.cc:302] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.034857 3008 raft_consensus.cc:1226] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Rejecting Update request from peer c93a6d22180a41a686c4556afd6a3e6b for earlier term 4. Current term is 5. Ops: [4.9-4.9]
I20240502 07:59:40.034891 4855 raft_consensus.cc:2781] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:40.034953 4855 raft_consensus.cc:686] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.034960 2944 leader_election.cc:302] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.035071 4855 consensus_queue.cc:237] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.035212 4885 raft_consensus.cc:2781] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.035274 4885 raft_consensus.cc:686] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.035393 4885 consensus_queue.cc:237] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 2.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.035554 4855 consensus_queue.cc:1035] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:40.035540 2944 leader_election.cc:302] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.036012 4883 raft_consensus.cc:2726] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240502 07:59:40.036011 2470 catalog_manager.cc:5484] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 3 to 4, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 4 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.036302 2944 leader_election.cc:302] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.036435 4853 raft_consensus.cc:2726] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20240502 07:59:40.038434 4781 heartbeater.cc:503] Master 127.15.129.190:34757 was elected leader, sending a full tablet report...
I20240502 07:59:40.039656 2467 catalog_manager.cc:5484] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 3 to 5, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 5 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
W20240502 07:59:40.041980 2707 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.046582 2745 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "512e1322925c415f8891dfe6621160e1"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.046716 2745 raft_consensus.cc:482] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.046787 2745 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.047757 2745 raft_consensus.cc:504] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.047969 2616 raft_consensus.cc:1261] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 4 index: 15. Preceding OpId from leader: term: 5 index: 16. (index mismatch)
I20240502 07:59:40.048419 2745 leader_election.cc:288] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 6 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.048429 4853 consensus_queue.cc:1035] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20240502 07:59:40.048548 4853 consensus_queue.cc:1225] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Peer c93a6d22180a41a686c4556afd6a3e6b log is divergent from this leader: its last log entry 4.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20240502 07:59:40.048873 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.048919 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.049005 3008 raft_consensus.cc:3032] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.049002 2877 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.049041 3008 raft_consensus.cc:729] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.049177 2745 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c67533803b9c496295f4f57e77a51374"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.049271 2745 raft_consensus.cc:482] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.049290 2744 raft_consensus.cc:1261] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 3 index: 14. Preceding OpId from leader: term: 5 index: 16. (index mismatch)
I20240502 07:59:40.049647 4855 consensus_queue.cc:1035] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 07:59:40.050101 2877 raft_consensus.cc:2454] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 6.
I20240502 07:59:40.049314 2745 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.050570 2682 leader_election.cc:302] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.051021 4193 raft_consensus.cc:2781] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.051093 2745 raft_consensus.cc:504] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.051251 2616 raft_consensus.cc:1203] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Deduplicated request from leader. Original: 3.13->[3.14-5.16] Dedup: 3.14->[5.15-5.16]
I20240502 07:59:40.051343 2616 pending_rounds.cc:76] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b: Aborting all ops after (but not including) 14
I20240502 07:59:40.051368 2616 pending_rounds.cc:97] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b: Aborting uncommitted NO_OP operation due to leader change: 4.15
I20240502 07:59:40.051396 2616 raft_consensus.cc:2866] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 07:59:40.051122 4193 raft_consensus.cc:686] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 6 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.051596 4193 consensus_queue.cc:237] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.051697 3008 consensus_queue.cc:260] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.051771 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.051846 3008 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.051846 3005 raft_consensus.cc:3032] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:40.052465 3005 raft_consensus.cc:729] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.052603 3005 consensus_queue.cc:260] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.052755 3005 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.052822 3008 raft_consensus.cc:2396] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20240502 07:59:40.053212 2745 leader_election.cc:288] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.053805 2744 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.053802 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.053896 2744 raft_consensus.cc:482] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.053902 2877 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.053951 2744 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.053920 2466 catalog_manager.cc:5484] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 5 to 6, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 6 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.054764 2744 raft_consensus.cc:504] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
W20240502 07:59:40.056565 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45992: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:40.057476 2877 raft_consensus.cc:2396] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 07:59:40.060730 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.060863 2877 raft_consensus.cc:3032] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:40.060916 2877 raft_consensus.cc:729] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.061058 2877 consensus_queue.cc:260] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 4.10, Last appended by leader: 10, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.061239 2877 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.061870 2744 leader_election.cc:288] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.062183 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.062309 2613 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.062466 2745 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.062579 2745 raft_consensus.cc:482] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.062637 2745 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.063344 2613 raft_consensus.cc:2454] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 5.
I20240502 07:59:40.063647 2682 leader_election.cc:302] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.063820 4193 raft_consensus.cc:2781] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.063896 4193 raft_consensus.cc:686] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.064054 2747 raft_consensus.cc:1226] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Rejecting Update request from peer c93a6d22180a41a686c4556afd6a3e6b for earlier term 4. Current term is 5. Ops: [4.9-4.9]
I20240502 07:59:40.064007 4193 consensus_queue.cc:237] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 4.10, Last appended by leader: 10, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.064478 2470 catalog_manager.cc:5484] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 4 to 5, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 5 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.064729 2470 catalog_manager.cc:5484] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 3 to 4, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 4 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.064925 2470 catalog_manager.cc:5484] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 4 to 5, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 5 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.065119 2470 catalog_manager.cc:5484] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 3 to 5, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 5 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.065338 2470 catalog_manager.cc:5484] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 4 to 5, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 5 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.065531 2470 catalog_manager.cc:5484] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 4 to 5, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 5 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.065703 2470 catalog_manager.cc:5484] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 3 to 4, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 4 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.065979 2466 catalog_manager.cc:5484] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 4 to 5, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 5 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.066150 2745 raft_consensus.cc:504] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.066726 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.066839 2878 raft_consensus.cc:3032] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:40.066882 2878 raft_consensus.cc:729] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.067095 2878 consensus_queue.cc:260] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 4.9, Last appended by leader: 9, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.067230 2878 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.069054 2878 raft_consensus.cc:2396] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 5 because replica has last-logged OpId of term: 4 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.069780 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.069909 2613 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.070971 2613 raft_consensus.cc:2454] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 5.
I20240502 07:59:40.071128 2745 leader_election.cc:288] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.071244 2682 leader_election.cc:302] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters: 606c439025f24e53a2feebeaf68c1681
I20240502 07:59:40.071625 4193 raft_consensus.cc:2781] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.071724 4193 raft_consensus.cc:686] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.071849 4193 consensus_queue.cc:237] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.072640 2745 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f116334237934021a25a6cacae430034"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.072727 2745 raft_consensus.cc:482] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.072773 2745 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.073978 2466 catalog_manager.cc:5484] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 4 to 5, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 5 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.073776 2745 raft_consensus.cc:504] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.075117 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.075233 3008 raft_consensus.cc:3032] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:40.075274 3008 raft_consensus.cc:729] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.075417 3008 consensus_queue.cc:260] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 4.9, Last appended by leader: 9, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.075572 3008 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
W20240502 07:59:40.077075 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.077704 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.077822 2877 raft_consensus.cc:2454] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 5.
I20240502 07:59:40.077845 2613 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.078822 2613 raft_consensus.cc:2454] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 5.
I20240502 07:59:40.078987 3008 raft_consensus.cc:2396] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 5 because replica has last-logged OpId of term: 4 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.079018 2745 leader_election.cc:288] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.079201 2682 leader_election.cc:302] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.079366 3005 raft_consensus.cc:2396] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 07:59:40.079425 4906 raft_consensus.cc:2781] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.079499 4906 raft_consensus.cc:686] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.079617 4906 consensus_queue.cc:237] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.079699 2681 leader_election.cc:302] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1; no voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.079931 4922 raft_consensus.cc:2726] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20240502 07:59:40.080901 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.081007 2747 raft_consensus.cc:482] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.081068 2747 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
W20240502 07:59:40.081146 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34814: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.082079 2747 raft_consensus.cc:504] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.082505 2467 catalog_manager.cc:5484] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 4 to 5, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 5 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.082854 2747 leader_election.cc:288] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.083428 2745 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.083536 2745 raft_consensus.cc:482] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.083601 2745 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.084676 2745 raft_consensus.cc:504] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.085214 2745 leader_election.cc:288] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 6 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.085942 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.086006 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 6 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.086050 2747 raft_consensus.cc:482] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.086103 2747 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.086130 2613 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.087029 2613 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 6.
I20240502 07:59:40.087029 2747 raft_consensus.cc:504] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.087342 2682 leader_election.cc:302] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.087440 2747 leader_election.cc:288] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.087658 4193 raft_consensus.cc:2781] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.087754 4193 raft_consensus.cc:686] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 6 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.087881 4193 consensus_queue.cc:237] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 5.17, Last appended by leader: 17, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.088421 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.088564 2878 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.089104 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.089218 2613 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.090209 2613 raft_consensus.cc:2396] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20240502 07:59:40.090760 2467 catalog_manager.cc:5484] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 5 to 6, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 6 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.091503 2613 raft_consensus.cc:1226] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Rejecting Update request from peer f1e12e7015044fbeb23a7cd6f4023bef for earlier term 5. Current term is 6. Ops: [5.17-5.17]
I20240502 07:59:40.091904 4855 consensus_queue.cc:1046] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: INVALID_TERM, Last received: 5.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 07:59:40.092168 4853 raft_consensus.cc:3032] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.092229 4853 raft_consensus.cc:729] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.092370 4853 consensus_queue.cc:260] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 5.17, Last appended by leader: 17, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.092670 4853 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.092985 2878 raft_consensus.cc:2454] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 5.
I20240502 07:59:40.093415 2682 leader_election.cc:302] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.094000 4193 raft_consensus.cc:2781] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.094101 4193 raft_consensus.cc:686] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.094228 4193 consensus_queue.cc:237] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.094313 2745 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.094403 2745 raft_consensus.cc:482] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.094462 2745 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.095394 2745 raft_consensus.cc:504] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.096182 2467 catalog_manager.cc:5484] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 3 to 5, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 5 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.096880 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.097004 2877 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.098096 2877 raft_consensus.cc:2396] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 5 because replica has last-logged OpId of term: 3 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 11.
I20240502 07:59:40.098675 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 6 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.099633 3008 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 6.
I20240502 07:59:40.099927 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.100034 3005 raft_consensus.cc:3032] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:40.100087 3005 raft_consensus.cc:729] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.100229 3005 consensus_queue.cc:260] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 4.12, Last appended by leader: 12, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.100386 3005 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.100350 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.100466 2613 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.101478 3005 raft_consensus.cc:2396] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 5 because replica has last-logged OpId of term: 4 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 11.
I20240502 07:59:40.101450 2613 raft_consensus.cc:2454] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 5.
I20240502 07:59:40.101763 2681 leader_election.cc:302] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1; no voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.101984 4906 raft_consensus.cc:2726] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20240502 07:59:40.101763 2682 leader_election.cc:302] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.102195 4906 raft_consensus.cc:2781] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.102278 4906 raft_consensus.cc:686] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.102422 4906 consensus_queue.cc:237] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 4.18, Last appended by leader: 18, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.104038 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.104172 2877 raft_consensus.cc:3032] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:40.104220 2877 raft_consensus.cc:729] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.104254 2467 catalog_manager.cc:5484] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 4 to 5, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 5 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.104386 2877 consensus_queue.cc:260] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 4.18, Last appended by leader: 18, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.105785 2877 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.106868 2877 raft_consensus.cc:2454] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 5.
I20240502 07:59:40.107095 2745 leader_election.cc:288] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.107707 2745 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.107821 2745 raft_consensus.cc:482] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.107872 2745 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.108462 2877 raft_consensus.cc:1261] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20240502 07:59:40.108806 4193 consensus_queue.cc:1035] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:40.109594 2616 raft_consensus.cc:1261] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20240502 07:59:40.110169 4193 consensus_queue.cc:1035] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:40.110280 4193 consensus_queue.cc:1225] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Peer c93a6d22180a41a686c4556afd6a3e6b log is divergent from this leader: its last log entry 3.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 07:59:40.110973 2616 pending_rounds.cc:76] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b: Aborting all ops after (but not including) 3
I20240502 07:59:40.111025 2616 pending_rounds.cc:97] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20240502 07:59:40.111064 2616 raft_consensus.cc:2866] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 07:59:40.112790 2745 raft_consensus.cc:504] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.113440 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.113565 2878 raft_consensus.cc:3032] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:40.113610 2878 raft_consensus.cc:729] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.113755 2878 consensus_queue.cc:260] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 4.9, Last appended by leader: 9, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.113922 2878 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.114918 2878 raft_consensus.cc:2454] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 5.
I20240502 07:59:40.117367 2745 leader_election.cc:288] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.117457 4839 mvcc.cc:205] Tried to move back new op lower bound from 7023152251312349184 to 7023152248060276736. Current Snapshot: MvccSnapshot[applied={T|T < 7023152251312349184}]
I20240502 07:59:40.118280 2745 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.118394 2745 raft_consensus.cc:482] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.118450 2745 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.119664 2745 raft_consensus.cc:504] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.120371 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.120507 3005 raft_consensus.cc:3032] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:40.120561 3005 raft_consensus.cc:729] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.120748 3005 consensus_queue.cc:260] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 4.9, Last appended by leader: 9, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.120970 3005 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.121371 2682 leader_election.cc:302] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.121563 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.121665 4922 raft_consensus.cc:2781] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.121752 4922 raft_consensus.cc:686] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.121773 2616 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.121881 4922 consensus_queue.cc:237] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 4.9, Last appended by leader: 9, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.122932 2616 raft_consensus.cc:2454] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 5.
I20240502 07:59:40.123991 2467 catalog_manager.cc:5484] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 4 to 5, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 5 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.124936 3008 raft_consensus.cc:1261] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20240502 07:59:40.125211 2878 raft_consensus.cc:1261] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20240502 07:59:40.125681 4193 consensus_queue.cc:1035] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:40.125789 4193 consensus_queue.cc:1225] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Peer f1e12e7015044fbeb23a7cd6f4023bef log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20240502 07:59:40.125844 3005 raft_consensus.cc:2396] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 5 because replica has last-logged OpId of term: 4 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.126142 4193 consensus_queue.cc:1035] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20240502 07:59:40.126749 3008 pending_rounds.cc:76] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef: Aborting all ops after (but not including) 3
I20240502 07:59:40.126792 3008 pending_rounds.cc:97] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef: Aborting uncommitted NO_OP operation due to leader change: 2.4
I20240502 07:59:40.126816 3008 raft_consensus.cc:2866] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 07:59:40.126842 3008 pending_rounds.cc:97] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20240502 07:59:40.126855 3008 raft_consensus.cc:2866] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20240502 07:59:40.130817 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60784: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.130977 2745 leader_election.cc:288] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.131525 2745 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.131645 2745 raft_consensus.cc:482] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.131717 2745 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.132861 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.132941 2745 raft_consensus.cc:504] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.133196 2616 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.133569 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 6 candidate_status { last_received { term: 4 index: 25 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.133697 3005 raft_consensus.cc:3032] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.133744 3005 raft_consensus.cc:729] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.133929 3005 consensus_queue.cc:260] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 5.26, Last appended by leader: 26, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.134095 3005 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.134582 2616 raft_consensus.cc:2454] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 5.
I20240502 07:59:40.135008 3005 raft_consensus.cc:2396] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 6 because replica has last-logged OpId of term: 5 index: 26, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 25.
I20240502 07:59:40.135133 2682 leader_election.cc:302] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters: f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.135360 4193 raft_consensus.cc:2781] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.135450 4193 raft_consensus.cc:686] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.135596 4193 consensus_queue.cc:237] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 3.8, Last appended by leader: 8, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.137080 2467 catalog_manager.cc:5484] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 4 to 5, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 5 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.138962 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 6 candidate_status { last_received { term: 4 index: 25 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.139096 2878 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.140388 2878 raft_consensus.cc:2396] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 6 because replica has last-logged OpId of term: 4 index: 26, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 25.
I20240502 07:59:40.140597 2745 leader_election.cc:288] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 6 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
W20240502 07:59:40.140640 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33762: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.140753 2682 leader_election.cc:302] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1; no voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.141055 4193 raft_consensus.cc:2726] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 07:59:40.141577 2745 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.141705 2745 raft_consensus.cc:482] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.141762 2745 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:40.142455 2745 raft_consensus.cc:504] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.142774 2745 leader_election.cc:288] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 4 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.143219 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 4 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.143230 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 4 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.143344 3005 raft_consensus.cc:3032] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 3 LEADER]: Stepping down as leader of term 3
I20240502 07:59:40.143405 3005 raft_consensus.cc:729] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.143406 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.143507 2747 raft_consensus.cc:482] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.143563 2747 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.143563 3005 consensus_queue.cc:260] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 3.23, Last appended by leader: 23, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.143536 2616 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:40.143747 3005 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 3 FOLLOWER]: Advancing to term 4
I20240502 07:59:40.144488 2747 raft_consensus.cc:504] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.144722 3005 raft_consensus.cc:2454] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 4.
I20240502 07:59:40.144865 2747 leader_election.cc:288] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.145107 2616 raft_consensus.cc:2454] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 4.
I20240502 07:59:40.145213 2681 leader_election.cc:302] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.145402 4193 raft_consensus.cc:2781] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Leader election won for term 4
I20240502 07:59:40.145501 4193 raft_consensus.cc:686] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 4 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.145658 4193 consensus_queue.cc:237] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 3.23, Last appended by leader: 23, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.146646 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.146709 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.146755 2747 raft_consensus.cc:482] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.146812 2747 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
W20240502 07:59:40.147195 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34814: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.146821 2878 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.147651 2747 raft_consensus.cc:504] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.147967 2747 leader_election.cc:288] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.148094 2878 raft_consensus.cc:2454] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 5.
I20240502 07:59:40.148272 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.148339 2682 leader_election.cc:302] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.148381 2877 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.148488 4193 raft_consensus.cc:2781] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.148571 4193 raft_consensus.cc:686] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.148684 4193 consensus_queue.cc:237] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.149411 2877 raft_consensus.cc:2396] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 07:59:40.149621 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.149724 2747 raft_consensus.cc:482] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.149784 2747 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.149791 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.149914 3005 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.150333 2467 catalog_manager.cc:5484] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 3 to 4, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 4 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.150847 3005 raft_consensus.cc:2454] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 5.
I20240502 07:59:40.150847 2747 raft_consensus.cc:504] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.151124 2681 leader_election.cc:302] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.151250 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.151433 2616 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.151703 4193 raft_consensus.cc:2781] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.151798 4193 raft_consensus.cc:686] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.151916 4193 consensus_queue.cc:237] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
W20240502 07:59:40.152293 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34834: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.152668 2616 raft_consensus.cc:2396] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 07:59:40.153277 2747 leader_election.cc:288] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 6 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.153834 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 6 candidate_status { last_received { term: 5 index: 12 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.153944 2616 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.153975 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.154064 2747 raft_consensus.cc:482] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.154112 2747 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.154646 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 6 candidate_status { last_received { term: 5 index: 12 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.154808 3005 raft_consensus.cc:3032] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.154865 2616 raft_consensus.cc:2454] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 6.
I20240502 07:59:40.154901 3005 raft_consensus.cc:729] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.154974 2747 raft_consensus.cc:504] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.155131 2682 leader_election.cc:302] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.155287 2747 leader_election.cc:288] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 6 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.155290 4906 raft_consensus.cc:2781] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.155270 3005 consensus_queue.cc:260] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 5.12, Last appended by leader: 12, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.155416 4906 raft_consensus.cc:686] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 6 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.155526 3005 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.155532 4906 consensus_queue.cc:237] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 5.12, Last appended by leader: 12, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
W20240502 07:59:40.155643 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33762: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.156471 3005 raft_consensus.cc:2454] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 6.
W20240502 07:59:40.156618 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.156553 2466 catalog_manager.cc:5484] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 3 to 5, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 5 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.156836 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 6 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.156810 2466 catalog_manager.cc:5484] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 3 to 5, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 5 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.156932 2877 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.157617 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 6 candidate_status { last_received { term: 3 index: 11 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.157728 3008 raft_consensus.cc:3032] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.157776 3008 raft_consensus.cc:729] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.157922 3008 consensus_queue.cc:260] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 5.12, Last appended by leader: 12, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.157999 2877 raft_consensus.cc:2454] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 6.
I20240502 07:59:40.158082 3008 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.158838 2682 leader_election.cc:302] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.158888 3008 raft_consensus.cc:2396] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 6 because replica has last-logged OpId of term: 5 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 11.
I20240502 07:59:40.159016 4193 raft_consensus.cc:2781] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.159107 4193 raft_consensus.cc:686] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 6 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.159233 4193 consensus_queue.cc:237] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 3.11, Last appended by leader: 11, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.159575 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d7b590b4226443d4899f3005840fd736"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.160071 2747 raft_consensus.cc:482] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.160133 2747 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.161132 2466 catalog_manager.cc:5484] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 5 to 6, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 6 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.161422 2466 catalog_manager.cc:5484] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 5 to 6, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 6 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.162505 2747 raft_consensus.cc:504] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
W20240502 07:59:40.162576 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.162860 2747 leader_election.cc:288] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 6 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.163514 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 6 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.163594 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 6 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.163641 3008 raft_consensus.cc:3032] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.163693 3008 raft_consensus.cc:729] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.163729 2877 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.163831 3008 consensus_queue.cc:260] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 5.11, Last appended by leader: 11, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.163975 3008 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.164747 3008 raft_consensus.cc:2396] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 6 because replica has last-logged OpId of term: 5 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 10.
I20240502 07:59:40.164781 2877 raft_consensus.cc:2396] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 6 because replica has last-logged OpId of term: 4 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 10.
W20240502 07:59:40.165196 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60784: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.165232 2681 leader_election.cc:302] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1; no voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.165385 4906 raft_consensus.cc:2726] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 07:59:40.165681 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.165788 2747 raft_consensus.cc:482] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.165834 2747 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.166482 2747 raft_consensus.cc:504] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.166793 2747 leader_election.cc:288] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 6 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.167089 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.167193 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.167317 2616 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.167361 3008 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.167528 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.167637 2747 raft_consensus.cc:482] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.167691 2747 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.168428 2616 raft_consensus.cc:2396] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20240502 07:59:40.168442 3008 raft_consensus.cc:2454] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 6.
I20240502 07:59:40.168594 2747 raft_consensus.cc:504] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.168745 2681 leader_election.cc:302] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.168931 4906 raft_consensus.cc:2781] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.169061 4906 raft_consensus.cc:686] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 6 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.169178 4906 consensus_queue.cc:237] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.169618 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 2 index: 16 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.169724 2616 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.169981 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 2 index: 16 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.170159 2877 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.170703 2616 raft_consensus.cc:2396] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 5 because replica has last-logged OpId of term: 3 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 16.
I20240502 07:59:40.170929 2747 leader_election.cc:288] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
W20240502 07:59:40.171124 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.171682 2877 raft_consensus.cc:2454] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 5.
I20240502 07:59:40.171911 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.172024 2747 raft_consensus.cc:482] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.172078 2747 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.172276 2682 leader_election.cc:302] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.172259 2466 catalog_manager.cc:5484] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 4 to 6, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 6 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.172564 4906 raft_consensus.cc:2781] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.172641 4906 raft_consensus.cc:686] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.172760 4906 consensus_queue.cc:237] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 2.16, Last appended by leader: 16, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.173085 2747 raft_consensus.cc:504] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.173429 2747 leader_election.cc:288] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.174072 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "811167eb170b49219b6cd22d865af334"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.174175 2747 raft_consensus.cc:482] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.174224 2747 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
W20240502 07:59:40.174328 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34814: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.175118 2747 raft_consensus.cc:504] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.175194 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.175324 2878 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.175928 2747 leader_election.cc:288] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 6 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.176234 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 6 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.176313 2878 raft_consensus.cc:2454] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 5.
I20240502 07:59:40.176368 3008 raft_consensus.cc:3032] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.176411 3008 raft_consensus.cc:729] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.176574 3008 consensus_queue.cc:260] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.9, Last appended by leader: 9, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.176717 3008 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.176901 2682 leader_election.cc:302] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.176932 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 6 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.177042 2613 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.177069 4193 raft_consensus.cc:2781] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.177137 4193 raft_consensus.cc:686] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.177273 4193 consensus_queue.cc:237] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 2.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.177757 3008 raft_consensus.cc:2396] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 6 because replica has last-logged OpId of term: 5 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 8.
I20240502 07:59:40.178205 2613 raft_consensus.cc:2396] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 6 because replica has last-logged OpId of term: 4 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 8.
I20240502 07:59:40.178367 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 5 candidate_status { last_received { term: 2 index: 7 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.178475 2616 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.178856 2466 catalog_manager.cc:5484] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 3 to 5, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 5 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.179461 2616 raft_consensus.cc:2396] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 5 because replica has last-logged OpId of term: 3 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 7.
I20240502 07:59:40.180073 2682 leader_election.cc:302] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1; no voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.180400 4193 raft_consensus.cc:2726] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 07:59:40.180521 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.180626 2747 raft_consensus.cc:482] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.180673 2747 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
W20240502 07:59:40.180955 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34834: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.181661 2747 raft_consensus.cc:504] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.181972 2747 leader_election.cc:288] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 6 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.182329 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.182441 2877 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.182755 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.182929 3008 raft_consensus.cc:3032] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.182871 2467 catalog_manager.cc:5484] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 3 to 5, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 5 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.183005 3008 raft_consensus.cc:729] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.183171 3008 consensus_queue.cc:260] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.183355 3008 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.183482 2877 raft_consensus.cc:2454] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 6.
I20240502 07:59:40.183745 2682 leader_election.cc:302] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.183893 4193 raft_consensus.cc:2781] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.183971 4193 raft_consensus.cc:686] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 6 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.184084 4193 consensus_queue.cc:237] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.184274 3008 raft_consensus.cc:2454] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 6.
W20240502 07:59:40.186035 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33762: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.187618 2466 catalog_manager.cc:5484] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 5 to 6, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 6 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.188014 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "18912e0059054584a990be7ccb4a5771"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.188149 2616 raft_consensus.cc:482] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.188217 2616 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
W20240502 07:59:40.189307 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.189313 2616 raft_consensus.cc:504] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.190090 2616 leader_election.cc:288] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 5 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.190222 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.190378 2877 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.190725 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f116334237934021a25a6cacae430034"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.190837 2616 raft_consensus.cc:482] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.190899 2616 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.191218 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.191339 3008 raft_consensus.cc:3032] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:40.191377 3008 raft_consensus.cc:729] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.191521 3008 consensus_queue.cc:260] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 4.13, Last appended by leader: 13, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.191671 3008 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.191942 2877 raft_consensus.cc:2396] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 5 because replica has last-logged OpId of term: 3 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 07:59:40.191979 2616 raft_consensus.cc:504] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.192620 3008 raft_consensus.cc:2396] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 5 because replica has last-logged OpId of term: 4 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 07:59:40.192616 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.192767 3005 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.192919 2550 leader_election.cc:302] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b; no voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.192955 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.193066 2747 raft_consensus.cc:3032] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.193102 2747 raft_consensus.cc:729] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.193150 4910 raft_consensus.cc:2726] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20240502 07:59:40.193290 2747 consensus_queue.cc:260] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.9, Last appended by leader: 9, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.193464 2747 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.193743 3005 raft_consensus.cc:2396] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 6 because replica has last-logged OpId of term: 4 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.194231 2616 leader_election.cc:288] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.194507 2747 raft_consensus.cc:2396] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 6 because replica has last-logged OpId of term: 5 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.194903 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.195016 2613 raft_consensus.cc:482] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.195005 2553 leader_election.cc:302] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b; no voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.195111 2613 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.195380 4910 raft_consensus.cc:2726] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 07:59:40.195960 2613 raft_consensus.cc:504] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.196249 2613 leader_election.cc:288] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.196619 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.196727 2747 raft_consensus.cc:3032] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.196862 2747 raft_consensus.cc:729] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.197234 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.197352 2613 raft_consensus.cc:482] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.197402 2613 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
W20240502 07:59:40.197563 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60784: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.198011 2747 consensus_queue.cc:260] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 5.11, Last appended by leader: 11, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.198173 2747 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.198302 2613 raft_consensus.cc:504] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.198628 2613 leader_election.cc:288] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.199077 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.199195 2878 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.199855 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.199976 3005 raft_consensus.cc:3032] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.200017 3005 raft_consensus.cc:729] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.200241 2878 raft_consensus.cc:2454] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 6.
I20240502 07:59:40.200277 3005 consensus_queue.cc:260] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 5.11, Last appended by leader: 11, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.200423 3005 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
W20240502 07:59:40.201186 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.201805 2551 leader_election.cc:302] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:40.201977 4910 raft_consensus.cc:2781] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.202052 4910 raft_consensus.cc:686] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.202095 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.202183 4910 consensus_queue.cc:237] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 4.10, Last appended by leader: 10, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.202190 2613 raft_consensus.cc:482] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.202312 2613 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.203377 2613 raft_consensus.cc:504] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.203851 2613 leader_election.cc:288] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 7 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.204250 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 12 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.204380 2745 raft_consensus.cc:3032] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.204418 2745 raft_consensus.cc:729] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.204548 2745 consensus_queue.cc:260] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 6.13, Last appended by leader: 13, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.204670 2745 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.204744 2466 catalog_manager.cc:5484] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 5 to 6, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 6 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.205173 3005 raft_consensus.cc:2396] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 6 because replica has last-logged OpId of term: 5 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 10.
I20240502 07:59:40.205662 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.205734 2745 raft_consensus.cc:2396] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 7 because replica has last-logged OpId of term: 6 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 12.
I20240502 07:59:40.205852 2877 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.207260 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.207556 2613 raft_consensus.cc:482] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.207638 2613 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
W20240502 07:59:40.208633 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34814: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.208854 2613 raft_consensus.cc:504] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.209504 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 12 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.210258 3005 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.210405 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.210510 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.210558 2878 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.210636 2745 raft_consensus.cc:3032] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.210686 2745 raft_consensus.cc:729] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.210844 2745 consensus_queue.cc:260] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.9, Last appended by leader: 9, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.210990 2745 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.211467 2613 leader_election.cc:288] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.211844 2878 raft_consensus.cc:2396] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 6 because replica has last-logged OpId of term: 4 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.212589 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "236a4d787879442aaff6983243f21f85"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.212761 2613 raft_consensus.cc:482] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
W20240502 07:59:40.212836 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34834: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.212875 2613 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.213356 2747 raft_consensus.cc:2396] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 6 because replica has last-logged OpId of term: 5 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 10.
I20240502 07:59:40.214118 2745 raft_consensus.cc:2396] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 6 because replica has last-logged OpId of term: 5 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.214104 3005 raft_consensus.cc:2454] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 7.
I20240502 07:59:40.214437 2550 leader_election.cc:302] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters: e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.214524 2553 leader_election.cc:302] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b; no voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.214735 4074 raft_consensus.cc:2781] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.214756 4910 raft_consensus.cc:2726] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 07:59:40.214838 4074 raft_consensus.cc:686] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 7 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.214994 4074 consensus_queue.cc:237] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 5.12, Last appended by leader: 12, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.215798 2613 raft_consensus.cc:504] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.216166 2613 leader_election.cc:288] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.216475 2877 raft_consensus.cc:2454] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 6.
I20240502 07:59:40.216949 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 20 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.217063 3005 raft_consensus.cc:3032] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.217104 3005 raft_consensus.cc:729] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.217276 3005 consensus_queue.cc:260] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 5.21, Last appended by leader: 21, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.217512 3005 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.217651 2466 catalog_manager.cc:5484] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 6 to 7, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 7 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.220398 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 20 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.220516 2878 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
W20240502 07:59:40.220733 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33762: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.221596 2551 leader_election.cc:302] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters: e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.221875 4074 raft_consensus.cc:2781] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.221975 4074 raft_consensus.cc:686] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.222043 2878 raft_consensus.cc:2396] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 6 because replica has last-logged OpId of term: 4 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 20.
I20240502 07:59:40.222123 4074 consensus_queue.cc:237] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 4.10, Last appended by leader: 10, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.221302 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.223146 2613 raft_consensus.cc:482] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.223212 2613 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
W20240502 07:59:40.225376 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.224300 2467 catalog_manager.cc:5484] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 5 to 6, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 6 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.226786 2613 raft_consensus.cc:504] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.218459 3005 raft_consensus.cc:2396] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 6 because replica has last-logged OpId of term: 5 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 20.
I20240502 07:59:40.227221 2550 leader_election.cc:302] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b; no voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.227506 4905 raft_consensus.cc:2726] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 07:59:40.228196 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.228319 2745 raft_consensus.cc:3032] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.228371 2745 raft_consensus.cc:729] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.228535 2745 consensus_queue.cc:260] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.228662 2745 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.229499 2745 raft_consensus.cc:2396] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20240502 07:59:40.229467 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.229640 3005 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.230432 3005 raft_consensus.cc:2454] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 7.
I20240502 07:59:40.230599 2613 leader_election.cc:288] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 7 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.230724 2550 leader_election.cc:302] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters: e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.230922 4905 raft_consensus.cc:2781] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.231000 4905 raft_consensus.cc:686] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 7 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.231114 4905 consensus_queue.cc:237] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.232621 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.232731 2613 raft_consensus.cc:482] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.232790 2613 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
W20240502 07:59:40.233155 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60784: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.233754 2613 raft_consensus.cc:504] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.234258 2613 leader_election.cc:288] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.234658 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.234758 2745 raft_consensus.cc:3032] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.234799 2745 raft_consensus.cc:729] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.234942 2745 consensus_queue.cc:260] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.235107 2745 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.234062 2467 catalog_manager.cc:5484] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 6 to 7, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 7 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.235004 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.235790 2877 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.236093 2745 raft_consensus.cc:2454] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 6.
I20240502 07:59:40.236909 2553 leader_election.cc:302] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.237074 4905 raft_consensus.cc:2781] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.237149 4905 raft_consensus.cc:686] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.237252 4905 consensus_queue.cc:237] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
W20240502 07:59:40.238751 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.238976 2877 raft_consensus.cc:2454] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 6.
I20240502 07:59:40.239737 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.239830 2613 raft_consensus.cc:482] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.239907 2613 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.240676 2467 catalog_manager.cc:5484] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 5 to 6, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 6 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.242020 2613 raft_consensus.cc:504] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.242619 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.242728 2745 raft_consensus.cc:3032] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.242767 2745 raft_consensus.cc:729] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.242961 2745 consensus_queue.cc:260] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 5.19, Last appended by leader: 19, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.243113 2745 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.243927 2745 raft_consensus.cc:2396] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 6 because replica has last-logged OpId of term: 5 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 18.
I20240502 07:59:40.244583 2613 leader_election.cc:288] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.244946 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.245074 2877 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.246058 2877 raft_consensus.cc:2454] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 6.
I20240502 07:59:40.246372 2551 leader_election.cc:302] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters: e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.246577 4074 raft_consensus.cc:2781] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.246656 4074 raft_consensus.cc:686] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.246780 4074 consensus_queue.cc:237] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 4.18, Last appended by leader: 18, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.247716 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.247848 2613 raft_consensus.cc:482] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.247946 2613 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
W20240502 07:59:40.248875 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34814: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.248905 2613 raft_consensus.cc:504] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.249694 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.249811 2745 raft_consensus.cc:3032] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.249861 2745 raft_consensus.cc:729] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.250018 2745 consensus_queue.cc:260] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 5.10, Last appended by leader: 10, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.250005 2467 catalog_manager.cc:5484] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 5 to 6, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 6 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.250160 2745 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.251286 2745 raft_consensus.cc:2396] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 6 because replica has last-logged OpId of term: 5 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 9.
I20240502 07:59:40.251817 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.251940 2877 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.252918 2877 raft_consensus.cc:2454] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 6.
I20240502 07:59:40.253069 2613 leader_election.cc:288] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.253203 2551 leader_election.cc:302] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters: e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.253458 4905 raft_consensus.cc:2781] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.253543 4905 raft_consensus.cc:686] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.253679 4905 consensus_queue.cc:237] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 4.9, Last appended by leader: 9, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
W20240502 07:59:40.254426 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34834: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.254719 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.254824 2613 raft_consensus.cc:482] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.254882 2613 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.255843 2613 raft_consensus.cc:504] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.256121 2613 leader_election.cc:288] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.256469 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.256569 2745 raft_consensus.cc:3032] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.256615 2745 raft_consensus.cc:729] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.256696 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.256757 2745 consensus_queue.cc:260] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.9, Last appended by leader: 9, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.256803 3005 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.256893 2745 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.256876 2467 catalog_manager.cc:5484] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 5 to 6, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 6 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.257858 3005 raft_consensus.cc:2396] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 6 because replica has last-logged OpId of term: 4 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.258592 2745 raft_consensus.cc:2396] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 6 because replica has last-logged OpId of term: 5 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.258832 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.258955 2613 raft_consensus.cc:482] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.259021 2613 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.259689 2613 raft_consensus.cc:504] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.260007 2613 leader_election.cc:288] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 5 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.260285 2553 leader_election.cc:302] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b; no voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.260371 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.260430 4905 raft_consensus.cc:2726] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 07:59:40.260471 3005 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.260663 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 5 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.260756 2745 raft_consensus.cc:3032] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 4 LEADER]: Stepping down as leader of term 4
I20240502 07:59:40.260797 2745 raft_consensus.cc:729] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.260970 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.260957 2745 consensus_queue.cc:260] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 4.24, Last appended by leader: 24, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.261056 2613 raft_consensus.cc:482] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.261101 2613 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.261096 2745 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.261504 3005 raft_consensus.cc:2454] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 5.
I20240502 07:59:40.261924 2613 raft_consensus.cc:504] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.261981 2745 raft_consensus.cc:2396] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 5 because replica has last-logged OpId of term: 4 index: 24, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 23.
I20240502 07:59:40.262079 2550 leader_election.cc:302] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.262234 4905 raft_consensus.cc:2781] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.262323 4905 raft_consensus.cc:686] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 5 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.262444 4905 consensus_queue.cc:237] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 3.23, Last appended by leader: 23, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.262773 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.262879 2877 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.263844 2877 raft_consensus.cc:2454] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 6.
I20240502 07:59:40.264267 2551 leader_election.cc:302] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:40.264461 4905 raft_consensus.cc:2781] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.264535 4905 raft_consensus.cc:686] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.264411 2467 catalog_manager.cc:5484] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 4 to 5, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 5 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.264642 4905 consensus_queue.cc:237] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.265444 2613 leader_election.cc:288] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.265658 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.265769 2745 raft_consensus.cc:3032] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.265851 2745 raft_consensus.cc:729] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.266026 2745 consensus_queue.cc:260] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.266083 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.266177 2616 raft_consensus.cc:482] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.266240 2616 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.266184 2745 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.267161 2616 raft_consensus.cc:504] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.267504 2616 leader_election.cc:288] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 7 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.267522 2745 raft_consensus.cc:2396] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20240502 07:59:40.268357 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.268456 2745 raft_consensus.cc:3032] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.268493 2745 raft_consensus.cc:729] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.268615 2745 consensus_queue.cc:260] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 6.18, Last appended by leader: 18, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
W20240502 07:59:40.268646 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33762: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.268726 2745 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.269237 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.269366 2616 raft_consensus.cc:482] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.269428 2616 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.269774 2745 raft_consensus.cc:2396] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 7 because replica has last-logged OpId of term: 6 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 16.
I20240502 07:59:40.270341 2616 raft_consensus.cc:504] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
W20240502 07:59:40.270536 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.270697 2616 leader_election.cc:288] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.271378 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.271490 3008 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.271729 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.271853 2877 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.272389 3008 raft_consensus.cc:2454] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 6.
I20240502 07:59:40.272671 2550 leader_election.cc:302] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.272903 2877 raft_consensus.cc:2454] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 6.
I20240502 07:59:40.272997 4905 raft_consensus.cc:2781] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.273083 4905 raft_consensus.cc:686] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.273202 4905 consensus_queue.cc:237] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 4.9, Last appended by leader: 9, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.273520 2467 catalog_manager.cc:5484] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 5 to 6, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 6 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.273663 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 7 candidate_status { last_received { term: 5 index: 16 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.273761 3005 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.274837 3005 raft_consensus.cc:2396] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 7 because replica has last-logged OpId of term: 5 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 16.
I20240502 07:59:40.275152 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.275281 2616 raft_consensus.cc:482] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.275349 2616 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.275808 2550 leader_election.cc:302] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b; no voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.276053 4905 raft_consensus.cc:2726] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 07:59:40.276364 2616 raft_consensus.cc:504] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.276754 2616 leader_election.cc:288] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.277088 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.277207 2877 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.277379 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.277482 3005 raft_consensus.cc:3032] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.277521 3005 raft_consensus.cc:729] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.277662 3005 consensus_queue.cc:260] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.277776 3005 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.278179 2877 raft_consensus.cc:2396] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 6 because replica has last-logged OpId of term: 4 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 07:59:40.278571 3005 raft_consensus.cc:2396] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 07:59:40.278910 2550 leader_election.cc:302] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b; no voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.279146 4905 raft_consensus.cc:2726] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 07:59:40.279661 2467 catalog_manager.cc:5484] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 4 to 6, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 6 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.279976 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.280093 2616 raft_consensus.cc:482] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.280157 2616 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.281069 2616 raft_consensus.cc:504] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.281787 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.281888 2877 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.281905 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 16 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.282174 2616 leader_election.cc:288] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.282034 2745 raft_consensus.cc:3032] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.282438 2745 raft_consensus.cc:729] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.282655 2745 consensus_queue.cc:260] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 5.17, Last appended by leader: 17, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.282809 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.282841 2745 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.282923 2616 raft_consensus.cc:482] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.282989 2616 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.283172 2877 raft_consensus.cc:2454] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 6.
I20240502 07:59:40.283452 2551 leader_election.cc:302] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:40.283653 4905 raft_consensus.cc:2781] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.283746 4905 raft_consensus.cc:686] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.283910 4905 consensus_queue.cc:237] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 3.16, Last appended by leader: 16, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.283957 2616 raft_consensus.cc:504] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.284718 2616 leader_election.cc:288] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.285022 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.285140 2747 raft_consensus.cc:3032] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.285195 2747 raft_consensus.cc:729] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.285362 2747 consensus_queue.cc:260] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.285413 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.285501 2616 raft_consensus.cc:482] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.285534 2747 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.285559 2616 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.286412 2616 raft_consensus.cc:504] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.286526 2747 raft_consensus.cc:2396] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 7.
I20240502 07:59:40.286739 2616 leader_election.cc:288] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 5 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
W20240502 07:59:40.287106 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60784: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.287840 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.287964 2878 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.288945 2878 raft_consensus.cc:2396] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 5 because replica has last-logged OpId of term: 3 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 10.
I20240502 07:59:40.288879 2467 catalog_manager.cc:5484] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 5 to 6, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 6 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
W20240502 07:59:40.288998 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.289400 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.289490 2877 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.290016 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.290129 2613 raft_consensus.cc:482] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.290179 2613 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.290328 2877 raft_consensus.cc:2454] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 6.
I20240502 07:59:40.290782 2551 leader_election.cc:302] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters: e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.290953 4074 raft_consensus.cc:2781] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.291036 4074 raft_consensus.cc:686] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.291134 2613 raft_consensus.cc:504] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.291170 4074 consensus_queue.cc:237] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.291577 2613 leader_election.cc:288] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.291863 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.291980 3008 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.292410 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 6 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.292508 2877 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.292950 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "811167eb170b49219b6cd22d865af334"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.293011 3008 raft_consensus.cc:2454] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 6.
I20240502 07:59:40.293072 2616 raft_consensus.cc:482] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.293134 2616 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.293526 2877 raft_consensus.cc:2396] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 6 because replica has last-logged OpId of term: 4 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 07:59:40.294054 2616 raft_consensus.cc:504] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.294353 2616 leader_election.cc:288] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 7 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.294746 2550 leader_election.cc:302] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681
I20240502 07:59:40.294751 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.294857 2747 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.294963 4905 raft_consensus.cc:2781] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.295037 4905 raft_consensus.cc:686] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.295150 4905 consensus_queue.cc:237] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 3.12, Last appended by leader: 12, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.295295 2467 catalog_manager.cc:5484] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 5 to 6, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 6 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.295889 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 5 candidate_status { last_received { term: 2 index: 10 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.296005 3005 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 4 FOLLOWER]: Advancing to term 5
I20240502 07:59:40.297116 3005 raft_consensus.cc:2454] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 5.
I20240502 07:59:40.297461 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.297566 3005 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.297564 2747 raft_consensus.cc:2454] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 7.
I20240502 07:59:40.297858 2553 leader_election.cc:302] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.297911 2745 raft_consensus.cc:2396] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 6 because replica has last-logged OpId of term: 5 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 16.
I20240502 07:59:40.298192 2550 leader_election.cc:302] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681
I20240502 07:59:40.298460 3005 raft_consensus.cc:2396] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 7 because replica has last-logged OpId of term: 5 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 9.
I20240502 07:59:40.298456 4074 raft_consensus.cc:2781] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.298552 4074 raft_consensus.cc:686] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 7 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.298666 4074 consensus_queue.cc:237] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 4.9, Last appended by leader: 9, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.299129 4074 raft_consensus.cc:2781] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Leader election won for term 5
I20240502 07:59:40.299207 4074 raft_consensus.cc:686] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 5 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.299315 4074 consensus_queue.cc:237] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 2.10, Last appended by leader: 10, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.303395 2467 catalog_manager.cc:5484] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 4 to 6, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 6 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.303658 2467 catalog_manager.cc:5484] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 3 to 5, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 5 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.303823 2467 catalog_manager.cc:5484] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 5 to 7, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 7 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.304733 2877 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "512e1322925c415f8891dfe6621160e1"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.304881 2877 raft_consensus.cc:482] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.304956 2877 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
W20240502 07:59:40.306703 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34814: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.306988 2877 raft_consensus.cc:504] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.307389 2877 leader_election.cc:288] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
W20240502 07:59:40.307437 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34834: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.307790 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.307910 2745 raft_consensus.cc:3032] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.307881 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.307952 2745 raft_consensus.cc:729] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.307991 3005 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.308100 2745 consensus_queue.cc:260] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.308277 2745 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.308368 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.308460 2878 raft_consensus.cc:482] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.308507 2878 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.309031 3005 raft_consensus.cc:2454] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 7.
I20240502 07:59:40.309345 2745 raft_consensus.cc:2454] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 7.
I20240502 07:59:40.309574 2812 leader_election.cc:302] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.309782 4993 raft_consensus.cc:2781] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.309860 4993 raft_consensus.cc:686] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.309978 4993 consensus_queue.cc:237] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.310669 2878 raft_consensus.cc:504] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.311014 2878 leader_election.cc:288] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.311305 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.311422 2745 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.311478 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.311594 2613 raft_consensus.cc:3032] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.311631 2613 raft_consensus.cc:729] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.311761 2613 consensus_queue.cc:260] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 6.11, Last appended by leader: 11, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.311882 2613 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.312378 2745 raft_consensus.cc:2396] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 5 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 10.
I20240502 07:59:40.312105 2467 catalog_manager.cc:5484] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 6 to 7, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 7 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.312889 2613 raft_consensus.cc:2396] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 6 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 10.
I20240502 07:59:40.313086 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.313201 2878 raft_consensus.cc:482] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.313261 2878 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.314077 2878 raft_consensus.cc:504] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.314441 2878 leader_election.cc:288] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.314460 2813 leader_election.cc:302] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.314824 4993 raft_consensus.cc:2726] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 07:59:40.315178 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.315305 3005 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.315277 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.315392 2613 raft_consensus.cc:3032] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.315436 2613 raft_consensus.cc:729] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.315439 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "18912e0059054584a990be7ccb4a5771"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.315533 2878 raft_consensus.cc:482] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.315579 2878 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.315564 2613 consensus_queue.cc:260] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 6.11, Last appended by leader: 11, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.315686 2613 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.316373 3005 raft_consensus.cc:2396] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 5 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 10.
I20240502 07:59:40.316462 2613 raft_consensus.cc:2396] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 6 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 10.
I20240502 07:59:40.316426 2878 raft_consensus.cc:504] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.316717 2878 leader_election.cc:288] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 6 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.316749 2813 leader_election.cc:302] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.317054 4993 raft_consensus.cc:2726] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 07:59:40.317032 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 6 candidate_status { last_received { term: 3 index: 13 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.317155 3005 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.317272 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.317412 2878 raft_consensus.cc:482] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.317473 2878 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.317492 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 6 candidate_status { last_received { term: 3 index: 13 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.317597 2613 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.318116 3005 raft_consensus.cc:2396] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 6 because replica has last-logged OpId of term: 4 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 13.
I20240502 07:59:40.318138 2878 raft_consensus.cc:504] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.318480 2878 leader_election.cc:288] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 6 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.318826 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 6 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.318874 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 6 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.318954 2745 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.318524 2613 raft_consensus.cc:2454] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 6.
I20240502 07:59:40.319105 3005 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.319252 2813 leader_election.cc:302] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters: f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.319489 4993 raft_consensus.cc:2781] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.319584 4993 raft_consensus.cc:686] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 6 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.319752 4993 consensus_queue.cc:237] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 3.13, Last appended by leader: 13, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.319957 2745 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 6.
I20240502 07:59:40.320343 3005 raft_consensus.cc:2396] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 6 because replica has last-logged OpId of term: 4 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 07:59:40.320701 2815 leader_election.cc:302] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters: f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.320925 4991 raft_consensus.cc:2781] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.320896 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.321007 4991 raft_consensus.cc:686] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 6 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.321019 2878 raft_consensus.cc:482] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.321069 2878 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.321144 4991 consensus_queue.cc:237] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 3.12, Last appended by leader: 12, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.321765 2467 catalog_manager.cc:5484] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 4 to 6, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 6 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.326490 2878 raft_consensus.cc:504] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.327262 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.327410 2613 raft_consensus.cc:3032] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.327453 2613 raft_consensus.cc:729] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.327616 2613 consensus_queue.cc:260] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.327605 2467 catalog_manager.cc:5484] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 4 to 6, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 6 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.327773 2613 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.328333 2878 leader_election.cc:288] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.328871 2613 raft_consensus.cc:2396] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20240502 07:59:40.328941 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.329071 2745 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
W20240502 07:59:40.329456 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33762: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.330065 2745 raft_consensus.cc:2454] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 7.
I20240502 07:59:40.330485 2815 leader_election.cc:302] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.330657 4993 raft_consensus.cc:2781] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.330722 4993 raft_consensus.cc:686] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.330832 4993 consensus_queue.cc:237] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.331619 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
W20240502 07:59:40.331728 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.331758 2878 raft_consensus.cc:482] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.331861 2878 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.332760 2878 raft_consensus.cc:504] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.333129 2878 leader_election.cc:288] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.333459 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.333504 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.333578 2613 raft_consensus.cc:3032] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.333613 2745 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.333626 2613 raft_consensus.cc:729] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.333748 2877 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.333843 2877 raft_consensus.cc:482] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.333808 2613 consensus_queue.cc:260] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 6.19, Last appended by leader: 19, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.333892 2877 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.333951 2613 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.334518 2745 raft_consensus.cc:2396] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 5 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 18.
I20240502 07:59:40.334828 2613 raft_consensus.cc:2396] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 6 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 18.
I20240502 07:59:40.334784 2877 raft_consensus.cc:504] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.335230 2877 leader_election.cc:288] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.335265 2813 leader_election.cc:302] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.335466 4993 raft_consensus.cc:2726] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 07:59:40.335527 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.335626 2745 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.335779 2879 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.335875 2879 raft_consensus.cc:482] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.335911 2879 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.336436 2745 raft_consensus.cc:2396] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 5 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 9.
I20240502 07:59:40.336522 2879 raft_consensus.cc:504] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.336831 2879 leader_election.cc:288] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 6 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.337210 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.337262 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.337411 3005 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.337482 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.337585 2745 raft_consensus.cc:3032] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.337626 2745 raft_consensus.cc:729] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.337625 2879 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.337711 2879 raft_consensus.cc:482] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.337754 2879 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.337771 2745 consensus_queue.cc:260] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.337913 2745 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.338460 3005 raft_consensus.cc:2454] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 6.
I20240502 07:59:40.338582 2879 raft_consensus.cc:504] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.338798 2812 leader_election.cc:302] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.338907 2879 leader_election.cc:288] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.339154 4993 raft_consensus.cc:2781] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.339179 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.339203 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.339237 4993 raft_consensus.cc:686] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 6 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.339279 3005 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.339293 2747 raft_consensus.cc:3032] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.339327 2747 raft_consensus.cc:729] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.339372 4993 consensus_queue.cc:237] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.339440 2747 consensus_queue.cc:260] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.339567 2747 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.339694 2467 catalog_manager.cc:5484] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 6 to 7, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 7 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.340090 3005 raft_consensus.cc:2454] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 7.
I20240502 07:59:40.337363 2613 raft_consensus.cc:3032] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.340494 2747 raft_consensus.cc:2396] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 6 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20240502 07:59:40.340495 2613 raft_consensus.cc:729] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.340728 2613 consensus_queue.cc:260] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.10, Last appended by leader: 10, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.340935 2613 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.341265 2879 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.341696 2879 raft_consensus.cc:482] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.341758 2879 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.341920 2745 raft_consensus.cc:2396] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20240502 07:59:40.342113 2812 leader_election.cc:302] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters: e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.342336 4991 raft_consensus.cc:2781] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.342407 4991 raft_consensus.cc:686] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.342504 4991 consensus_queue.cc:237] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.342578 2879 raft_consensus.cc:504] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.342857 2879 leader_election.cc:288] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.343544 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 26 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.343657 3005 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.343816 2879 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.344110 2879 raft_consensus.cc:482] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.344179 2879 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.345048 2613 raft_consensus.cc:2396] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 6 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 9.
I20240502 07:59:40.345371 3005 raft_consensus.cc:2396] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 5 index: 26, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 26.
I20240502 07:59:40.345407 2813 leader_election.cc:302] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.345655 4991 raft_consensus.cc:2726] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 07:59:40.345726 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 26 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.345829 2745 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.346151 2467 catalog_manager.cc:5484] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 5 to 6, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 6 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.346388 2467 catalog_manager.cc:5484] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 6 to 7, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 7 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.347066 2745 raft_consensus.cc:2454] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 7.
W20240502 07:59:40.347108 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60784: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.347311 2815 leader_election.cc:302] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters: f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.347599 4991 raft_consensus.cc:2781] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.347683 4991 raft_consensus.cc:686] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.347798 4991 consensus_queue.cc:237] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 4.26, Last appended by leader: 26, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.347689 2879 raft_consensus.cc:504] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.349283 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.349308 2879 leader_election.cc:288] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.349421 2613 raft_consensus.cc:3032] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.349754 2613 raft_consensus.cc:729] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
W20240502 07:59:40.350090 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.350286 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.350400 3005 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.350443 2879 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.350558 2879 raft_consensus.cc:482] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.350605 2879 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.351258 3005 raft_consensus.cc:2454] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 7.
I20240502 07:59:40.351338 2879 raft_consensus.cc:504] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.351653 2879 leader_election.cc:288] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.351738 2613 consensus_queue.cc:260] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.10, Last appended by leader: 10, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.351889 2613 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.351984 2812 leader_election.cc:302] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.352053 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.352142 4991 raft_consensus.cc:2781] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.352144 3005 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.352203 4991 raft_consensus.cc:686] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.352311 4991 consensus_queue.cc:237] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 4.9, Last appended by leader: 9, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.352952 3005 raft_consensus.cc:2396] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 5 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.353106 2613 raft_consensus.cc:2396] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 6 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 9.
I20240502 07:59:40.353657 2879 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d7b590b4226443d4899f3005840fd736"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.353758 2879 raft_consensus.cc:482] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.353765 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.353812 2879 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.353865 2745 raft_consensus.cc:3032] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.353901 2745 raft_consensus.cc:729] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.354048 2745 consensus_queue.cc:260] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 6.12, Last appended by leader: 12, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.354166 2745 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.354606 2879 raft_consensus.cc:504] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.354956 2879 leader_election.cc:288] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.355080 2745 raft_consensus.cc:2396] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 6 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.355602 2815 leader_election.cc:302] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.355746 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 11 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.355849 3005 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.356098 4993 raft_consensus.cc:2726] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 07:59:40.356112 2879 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c67533803b9c496295f4f57e77a51374"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.356216 2879 raft_consensus.cc:482] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.356259 2879 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.356786 3005 raft_consensus.cc:2396] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 5 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 11.
I20240502 07:59:40.356709 2467 catalog_manager.cc:5484] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 5 to 7, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 7 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.357074 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 11 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.357223 2745 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.358117 2745 raft_consensus.cc:2454] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 7.
I20240502 07:59:40.357213 2879 raft_consensus.cc:504] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.358556 2879 leader_election.cc:288] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 6 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.359028 2815 leader_election.cc:302] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters: f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.359057 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.359156 3005 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.359577 4993 raft_consensus.cc:2781] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.359658 4993 raft_consensus.cc:686] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.359789 2879 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.359925 2879 raft_consensus.cc:482] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.359948 3005 raft_consensus.cc:2396] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20240502 07:59:40.360049 2879 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.360031 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.360126 2745 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.359774 4993 consensus_queue.cc:237] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 4.11, Last appended by leader: 11, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.361061 2745 raft_consensus.cc:2454] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 6.
I20240502 07:59:40.361339 2815 leader_election.cc:302] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters: f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.361557 2879 raft_consensus.cc:504] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.361569 4993 raft_consensus.cc:2781] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.361773 4993 raft_consensus.cc:686] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 6 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.361886 4993 consensus_queue.cc:237] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.363230 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 6 candidate_status { last_received { term: 3 index: 10 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.363369 2613 raft_consensus.cc:3032] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.363415 2613 raft_consensus.cc:729] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.363375 2467 catalog_manager.cc:5484] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 6 to 7, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 7 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.363559 2613 consensus_queue.cc:260] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 5.11, Last appended by leader: 11, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.363587 2467 catalog_manager.cc:5484] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 5 to 7, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 7 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.363700 2613 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.364615 2613 raft_consensus.cc:2396] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 6 because replica has last-logged OpId of term: 5 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 10.
I20240502 07:59:40.366986 2879 leader_election.cc:288] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 6 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.367126 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 6 candidate_status { last_received { term: 3 index: 10 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.367236 3008 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.367265 2745 raft_consensus.cc:1261] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20240502 07:59:40.367622 4993 consensus_queue.cc:1035] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:40.368199 3008 raft_consensus.cc:2454] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 6.
I20240502 07:59:40.368701 2879 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.368818 2879 raft_consensus.cc:482] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.368885 2879 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.369787 2879 raft_consensus.cc:504] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.370241 2879 leader_election.cc:288] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.370718 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 12 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.370828 2613 raft_consensus.cc:3032] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.370872 2613 raft_consensus.cc:729] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.371012 2613 consensus_queue.cc:260] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 6.13, Last appended by leader: 13, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.371142 2613 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.371639 2812 leader_election.cc:302] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.371963 4993 raft_consensus.cc:2781] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Leader election won for term 6
I20240502 07:59:40.371971 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 12 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.372048 4993 raft_consensus.cc:686] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 6 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.372084 3008 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.372128 2613 raft_consensus.cc:2396] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 6 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 12.
I20240502 07:59:40.372164 4993 consensus_queue.cc:237] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 3.10, Last appended by leader: 10, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.372798 2879 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.372890 2879 raft_consensus.cc:482] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.372942 2879 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.373052 3008 raft_consensus.cc:2454] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 7.
I20240502 07:59:40.373821 2812 leader_election.cc:302] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.374029 4990 raft_consensus.cc:2781] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.374095 4990 raft_consensus.cc:686] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.374198 4990 consensus_queue.cc:237] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 4.12, Last appended by leader: 12, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.373814 2879 raft_consensus.cc:504] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.375432 2879 leader_election.cc:288] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.375510 3008 raft_consensus.cc:1261] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 4 index: 12. Preceding OpId from leader: term: 6 index: 14. (index mismatch)
I20240502 07:59:40.373184 2467 catalog_manager.cc:5484] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 4 to 6, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 6 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.375980 3005 raft_consensus.cc:1261] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20240502 07:59:40.376536 2744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.376659 2744 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.377676 2744 raft_consensus.cc:2396] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 07:59:40.377862 4988 consensus_queue.cc:1035] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:40.377959 4988 consensus_queue.cc:1225] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Peer f1e12e7015044fbeb23a7cd6f4023bef log is divergent from this leader: its last log entry 4.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20240502 07:59:40.378175 4988 consensus_queue.cc:1035] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20240502 07:59:40.378266 4988 consensus_queue.cc:1225] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Peer f1e12e7015044fbeb23a7cd6f4023bef log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20240502 07:59:40.378948 2879 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.378993 2747 raft_consensus.cc:1261] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 2 index: 11. Preceding OpId from leader: term: 6 index: 14. (index mismatch)
I20240502 07:59:40.379045 2879 raft_consensus.cc:482] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.379092 2879 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.379465 3008 pending_rounds.cc:76] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef: Aborting all ops after (but not including) 5
I20240502 07:59:40.379542 3008 pending_rounds.cc:97] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20240502 07:59:40.379587 3008 raft_consensus.cc:2866] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 07:59:40.379746 4991 consensus_queue.cc:1035] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20240502 07:59:40.381059 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.381186 2613 raft_consensus.cc:3032] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.381232 2613 raft_consensus.cc:729] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.381538 2879 raft_consensus.cc:504] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.381534 2613 consensus_queue.cc:260] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.381743 2613 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.382191 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.382314 2616 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.382798 2467 catalog_manager.cc:5484] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 5 to 6, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 6 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.383064 2467 catalog_manager.cc:5484] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 6 to 7, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 7 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.383317 2613 raft_consensus.cc:2396] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 07:59:40.383385 2616 raft_consensus.cc:2454] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 7.
I20240502 07:59:40.383684 2813 leader_election.cc:302] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.383903 2813 leader_election.cc:302] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:40.383991 3009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.384093 3009 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.384238 4988 raft_consensus.cc:2726] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 07:59:40.384308 4988 raft_consensus.cc:2781] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.384362 4988 raft_consensus.cc:686] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.384465 4988 consensus_queue.cc:237] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.384654 3005 pending_rounds.cc:76] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef: Aborting all ops after (but not including) 11
I20240502 07:59:40.384697 3005 pending_rounds.cc:97] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef: Aborting uncommitted NO_OP operation due to leader change: 4.12
I20240502 07:59:40.384752 3005 raft_consensus.cc:2866] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 07:59:40.385084 3009 raft_consensus.cc:2396] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 8.
W20240502 07:59:40.386844 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.388705 2879 leader_election.cc:288] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.393911 2747 raft_consensus.cc:1261] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20240502 07:59:40.394292 4990 consensus_queue.cc:1035] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20240502 07:59:40.394366 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45948: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:40.394811 3009 raft_consensus.cc:1261] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20240502 07:59:40.395166 4991 consensus_queue.cc:1035] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:40.397042 2879 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.397161 2879 raft_consensus.cc:482] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.397222 2879 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
W20240502 07:59:40.397678 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.399192 2879 raft_consensus.cc:504] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.399943 4869 mvcc.cc:205] Tried to move back new op lower bound from 7023152252488441856 to 7023152252150992896. Current Snapshot: MvccSnapshot[applied={T|T < 7023152252488441856 or (T in {7023152252488441856})}]
I20240502 07:59:40.400692 2475 catalog_manager.cc:5484] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 5 to 7, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 7 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.401105 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 2 index: 15 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.401201 2747 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.402648 2747 raft_consensus.cc:2396] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 5 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 15.
I20240502 07:59:40.402784 2879 leader_election.cc:288] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.403527 2879 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
W20240502 07:59:40.404214 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.404337 2879 raft_consensus.cc:482] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.404412 2879 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.405033 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 2 index: 15 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.405228 2613 raft_consensus.cc:3032] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.405349 2613 raft_consensus.cc:729] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.405561 2879 raft_consensus.cc:504] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.405623 2613 consensus_queue.cc:260] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 6.17, Last appended by leader: 17, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.405886 2613 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.406491 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.406589 2616 raft_consensus.cc:3032] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.406625 2616 raft_consensus.cc:729] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.406634 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.406725 2747 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.406757 2616 consensus_queue.cc:260] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.406886 2616 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.407850 2616 raft_consensus.cc:2396] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 07:59:40.408447 2879 leader_election.cc:288] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.409253 2879 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "236a4d787879442aaff6983243f21f85"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.409600 2879 raft_consensus.cc:482] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.409660 2879 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.410749 2747 raft_consensus.cc:2396] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 07:59:40.411500 2613 raft_consensus.cc:2396] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 6 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 15.
I20240502 07:59:40.411453 2879 raft_consensus.cc:504] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.411756 2815 leader_election.cc:302] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.412567 4990 raft_consensus.cc:2726] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 07:59:40.413100 2879 leader_election.cc:288] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.413614 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 21 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.413727 3008 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
W20240502 07:59:40.413986 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.414408 2879 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.414506 2879 raft_consensus.cc:482] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.414554 2879 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.414584 3008 raft_consensus.cc:2396] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 5 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 21.
I20240502 07:59:40.412842 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 21 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
W20240502 07:59:40.414570 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45992: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:40.415446 2616 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.416407 2616 raft_consensus.cc:2454] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 7.
I20240502 07:59:40.416699 2813 leader_election.cc:302] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.416896 4993 raft_consensus.cc:2726] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 07:59:40.416908 2813 leader_election.cc:302] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters: f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.417197 4990 raft_consensus.cc:2781] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.417299 4990 raft_consensus.cc:686] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.417460 4990 consensus_queue.cc:237] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 4.21, Last appended by leader: 21, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.417619 2879 raft_consensus.cc:504] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.418165 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.418267 2616 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.419126 2616 raft_consensus.cc:2454] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 7.
I20240502 07:59:40.419147 2475 catalog_manager.cc:5484] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 5 to 7, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 7 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.419481 2813 leader_election.cc:302] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:40.419710 2616 raft_consensus.cc:1261] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20240502 07:59:40.419713 4990 raft_consensus.cc:2781] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.419795 4990 raft_consensus.cc:686] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.419926 4990 consensus_queue.cc:237] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 4.9, Last appended by leader: 9, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.419960 2744 raft_consensus.cc:1261] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Refusing update from remote peer 606c439025f24e53a2feebeaf68c1681: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20240502 07:59:40.420435 4525 consensus_queue.cc:1035] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:40.421023 4993 consensus_queue.cc:1035] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20240502 07:59:40.421113 4993 consensus_queue.cc:1225] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Peer c93a6d22180a41a686c4556afd6a3e6b log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20240502 07:59:40.421169 2879 leader_election.cc:288] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 7 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
W20240502 07:59:40.421310 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45948: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:40.421856 2616 pending_rounds.cc:76] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b: Aborting all ops after (but not including) 6
I20240502 07:59:40.421911 2616 pending_rounds.cc:97] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20240502 07:59:40.421968 2616 raft_consensus.cc:2866] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 07:59:40.422885 2475 catalog_manager.cc:5484] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 5 to 7, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 7 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.425225 2744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.425369 2744 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.426400 2744 raft_consensus.cc:2396] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 7 because replica has last-logged OpId of term: 5 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 9.
W20240502 07:59:40.436645 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.439338 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "512e1322925c415f8891dfe6621160e1"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.439484 3005 raft_consensus.cc:482] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.439561 3005 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.440672 3005 raft_consensus.cc:504] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.441057 3005 leader_election.cc:288] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.442116 4451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:40.443539 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.443671 2879 raft_consensus.cc:3032] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.443715 2879 raft_consensus.cc:729] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.443778 2744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.443885 2744 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.443842 2879 consensus_queue.cc:260] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.444020 2879 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.444607 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.444729 3005 raft_consensus.cc:482] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.444783 3005 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.445087 2879 raft_consensus.cc:2454] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 8.
I20240502 07:59:40.445742 2744 raft_consensus.cc:2454] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 8.
I20240502 07:59:40.445760 3005 raft_consensus.cc:504] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.445920 2944 leader_election.cc:302] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.446115 3005 leader_election.cc:288] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.446622 4855 raft_consensus.cc:2781] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.446708 4855 raft_consensus.cc:686] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.446823 4855 consensus_queue.cc:237] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.447214 3008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.447345 3008 raft_consensus.cc:482] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.447397 3008 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.448071 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 11 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.448184 2616 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.448349 3008 raft_consensus.cc:504] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.448675 3008 leader_election.cc:288] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.448714 2467 catalog_manager.cc:5484] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 7 to 8, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 8 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.449172 2616 raft_consensus.cc:2396] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 8 because replica has last-logged OpId of term: 6 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 11.
I20240502 07:59:40.451373 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 11 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.451488 2879 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
W20240502 07:59:40.451849 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.452389 2879 raft_consensus.cc:2454] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 8.
I20240502 07:59:40.453377 2944 leader_election.cc:302] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.453558 4853 raft_consensus.cc:2781] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.453523 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.453634 4853 raft_consensus.cc:686] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.453650 2613 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.453768 4853 consensus_queue.cc:237] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 5.11, Last appended by leader: 11, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.453979 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.454093 2745 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.454615 2613 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 8.
I20240502 07:59:40.454926 2944 leader_election.cc:302] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.455070 4855 raft_consensus.cc:2781] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.455133 4855 raft_consensus.cc:686] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.455147 2745 raft_consensus.cc:2396] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 8 because replica has last-logged OpId of term: 6 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 17.
I20240502 07:59:40.455255 4855 consensus_queue.cc:237] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 5.17, Last appended by leader: 17, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.456054 2467 catalog_manager.cc:5484] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 6 to 8, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 8 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.456651 3008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "236a4d787879442aaff6983243f21f85"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.456750 3008 raft_consensus.cc:482] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.456808 3008 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.457875 3008 raft_consensus.cc:504] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.458226 3008 leader_election.cc:288] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
W20240502 07:59:40.458832 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34814: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.459475 3008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.459509 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 21 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.459584 3008 raft_consensus.cc:482] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.459610 2613 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.459637 3008 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.460441 3008 raft_consensus.cc:504] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.460527 2613 raft_consensus.cc:2454] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 8.
I20240502 07:59:40.460963 4584 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:40.460979 2944 leader_election.cc:302] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.461024 3008 leader_election.cc:288] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.461014 2475 catalog_manager.cc:5484] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 6 to 8, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 8 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.461233 4853 raft_consensus.cc:2781] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.461345 4853 raft_consensus.cc:686] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.461462 4853 consensus_queue.cc:237] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 5.21, Last appended by leader: 21, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.462203 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.462344 2745 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.463387 2745 raft_consensus.cc:2396] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 07:59:40.464165 4320 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:40.464566 3008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.464674 3008 raft_consensus.cc:482] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.464728 3008 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.464638 2475 catalog_manager.cc:5484] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 7 to 8, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 8 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.464995 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.465093 2613 raft_consensus.cc:3032] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.465127 2613 raft_consensus.cc:729] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.465258 2613 consensus_queue.cc:260] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
W20240502 07:59:40.465384 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45948: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:40.465430 2613 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.465687 3008 raft_consensus.cc:504] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.466265 3008 leader_election.cc:288] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 6 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.466483 2597 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:40.466703 2613 raft_consensus.cc:2396] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 07:59:40.466812 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 21 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.466926 2879 raft_consensus.cc:3032] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.466965 2879 raft_consensus.cc:729] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.467089 2879 consensus_queue.cc:260] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 7.22, Last appended by leader: 22, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.467212 2879 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.467300 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 6 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.467406 2745 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.467511 2944 leader_election.cc:302] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.467718 4853 raft_consensus.cc:2726] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240502 07:59:40.468163 2879 raft_consensus.cc:2396] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 8 because replica has last-logged OpId of term: 7 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 21.
I20240502 07:59:40.468359 2745 raft_consensus.cc:2396] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 6 because replica has last-logged OpId of term: 4 index: 24, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 23.
I20240502 07:59:40.469805 3008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.469903 3008 raft_consensus.cc:482] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.469997 3008 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.470750 3008 raft_consensus.cc:504] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.471926 3008 leader_election.cc:288] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.472258 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.472368 2745 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.472383 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.472484 2879 raft_consensus.cc:3032] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.472528 2879 raft_consensus.cc:729] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.472664 2879 consensus_queue.cc:260] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.472816 2879 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.473201 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "18912e0059054584a990be7ccb4a5771"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.473299 3005 raft_consensus.cc:482] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.473378 3005 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.473387 2745 raft_consensus.cc:2396] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 8 because replica has last-logged OpId of term: 6 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20240502 07:59:40.473812 2879 raft_consensus.cc:2396] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20240502 07:59:40.474081 2944 leader_election.cc:302] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.474292 4853 raft_consensus.cc:2726] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240502 07:59:40.474340 3005 raft_consensus.cc:504] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.474654 3005 leader_election.cc:288] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 7 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.474939 2989 tablet_service.cc:1429] Tablet server has 4 leaders and 0 scanners
I20240502 07:59:40.475163 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 7 candidate_status { last_received { term: 4 index: 13 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.475196 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 7 candidate_status { last_received { term: 4 index: 13 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.475291 2879 raft_consensus.cc:3032] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.475291 2613 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.475338 2879 raft_consensus.cc:729] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.475515 2879 consensus_queue.cc:260] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 6.14, Last appended by leader: 14, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.475580 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 6 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.475646 2879 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.475683 2616 raft_consensus.cc:3032] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 5 LEADER]: Stepping down as leader of term 5
I20240502 07:59:40.475724 2616 raft_consensus.cc:729] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.475863 2616 consensus_queue.cc:260] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 5.24, Last appended by leader: 24, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.475986 2616 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 5 FOLLOWER]: Advancing to term 6
I20240502 07:59:40.476260 2613 raft_consensus.cc:2454] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 7.
I20240502 07:59:40.476533 2879 raft_consensus.cc:2396] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 7 because replica has last-logged OpId of term: 6 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 13.
I20240502 07:59:40.476891 2944 leader_election.cc:302] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681
I20240502 07:59:40.476984 2616 raft_consensus.cc:2396] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 6 because replica has last-logged OpId of term: 5 index: 24, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 23.
I20240502 07:59:40.477037 4853 raft_consensus.cc:2781] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.477116 4853 raft_consensus.cc:686] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.477231 4853 consensus_queue.cc:237] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 4.13, Last appended by leader: 13, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.478015 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f116334237934021a25a6cacae430034"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.478152 3005 raft_consensus.cc:482] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.478228 3005 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.479332 2744 raft_consensus.cc:1261] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 11. (index mismatch)
I20240502 07:59:40.480250 4855 consensus_queue.cc:1035] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:40.481612 2475 catalog_manager.cc:5484] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 6 to 7, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 7 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.484239 2745 raft_consensus.cc:1261] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 6 index: 18. Preceding OpId from leader: term: 8 index: 19. (index mismatch)
I20240502 07:59:40.484723 4855 consensus_queue.cc:1035] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20240502 07:59:40.484825 4855 consensus_queue.cc:1225] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Peer e0a355821af5483f96aec7e128cc18e1 log is divergent from this leader: its last log entry 6.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20240502 07:59:40.485545 2744 pending_rounds.cc:76] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1: Aborting all ops after (but not including) 17
I20240502 07:59:40.485594 2744 pending_rounds.cc:97] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1: Aborting uncommitted NO_OP operation due to leader change: 6.18
I20240502 07:59:40.485626 2744 raft_consensus.cc:2866] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 07:59:40.489145 2944 leader_election.cc:302] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.491226 2727 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:40.493553 4883 raft_consensus.cc:2726] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240502 07:59:40.495018 2616 raft_consensus.cc:1261] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 5 index: 16. Preceding OpId from leader: term: 8 index: 19. (index mismatch)
I20240502 07:59:40.495810 4883 consensus_queue.cc:1035] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 07:59:40.496814 3005 raft_consensus.cc:504] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.497146 3005 leader_election.cc:288] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 7 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.498844 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.498942 3005 raft_consensus.cc:482] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.498994 3005 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.500046 3005 raft_consensus.cc:504] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.500793 3005 leader_election.cc:288] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 7 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.501201 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.501241 2879 raft_consensus.cc:1261] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 11. (index mismatch)
I20240502 07:59:40.501312 2747 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.501425 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.501528 2878 raft_consensus.cc:3032] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.501574 2878 raft_consensus.cc:729] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.501713 2878 consensus_queue.cc:260] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 6.14, Last appended by leader: 14, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.501860 2878 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.502365 2747 raft_consensus.cc:2396] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 7 because replica has last-logged OpId of term: 5 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 9.
I20240502 07:59:40.502790 2878 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 7.
I20240502 07:59:40.503510 2944 leader_election.cc:302] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.503629 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.503721 2747 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.504572 2747 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 7.
I20240502 07:59:40.504556 4855 consensus_queue.cc:1035] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20240502 07:59:40.504735 4883 raft_consensus.cc:2781] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.504819 4883 raft_consensus.cc:686] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.504949 4883 consensus_queue.cc:237] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 6.14, Last appended by leader: 14, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.506309 2616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.506458 2616 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.507483 2616 raft_consensus.cc:2454] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 7.
I20240502 07:59:40.508436 2467 catalog_manager.cc:5484] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 6 to 7, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 7 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.508989 2944 leader_election.cc:302] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters: e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.509343 4885 raft_consensus.cc:2781] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.509433 4885 raft_consensus.cc:686] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.509546 4885 consensus_queue.cc:237] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 4.9, Last appended by leader: 9, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.511456 2742 raft_consensus.cc:1261] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 6 index: 13. Preceding OpId from leader: term: 7 index: 14. (index mismatch)
I20240502 07:59:40.512389 4910 consensus_queue.cc:1035] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:40.512501 4910 consensus_queue.cc:1225] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Peer e0a355821af5483f96aec7e128cc18e1 log is divergent from this leader: its last log entry 6.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20240502 07:59:40.514120 2473 catalog_manager.cc:5484] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 5 to 7, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 7 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.515293 2747 pending_rounds.cc:76] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1: Aborting all ops after (but not including) 12
I20240502 07:59:40.515336 2747 pending_rounds.cc:97] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1: Aborting uncommitted NO_OP operation due to leader change: 6.13
I20240502 07:59:40.515363 2747 raft_consensus.cc:2866] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 07:59:40.521744 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.522025 3005 raft_consensus.cc:482] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.522091 3005 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.524900 3005 raft_consensus.cc:504] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.525264 3005 leader_election.cc:288] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.525846 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d7b590b4226443d4899f3005840fd736"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.525946 3010 raft_consensus.cc:482] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.526000 3010 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.526777 2859 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:40.526965 3010 raft_consensus.cc:504] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.527274 3010 leader_election.cc:288] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.528117 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c67533803b9c496295f4f57e77a51374"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.528218 3010 raft_consensus.cc:482] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.528271 3010 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.528262 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 12 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.528379 2747 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.529127 3010 raft_consensus.cc:504] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.529376 2747 raft_consensus.cc:2396] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 8 because replica has last-logged OpId of term: 6 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 12.
I20240502 07:59:40.529527 3010 leader_election.cc:288] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 7 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.530009 2742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 11 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.530109 2742 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.530982 2742 raft_consensus.cc:2454] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 8.
I20240502 07:59:40.533571 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.533710 3010 raft_consensus.cc:482] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.533826 3010 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.534711 3010 raft_consensus.cc:504] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.535540 3010 leader_election.cc:288] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 7 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.536087 3009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.536195 3009 raft_consensus.cc:482] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.536253 3009 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.537153 3009 raft_consensus.cc:504] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.537356 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 7 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.537483 3009 leader_election.cc:288] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.537487 2614 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.537778 3010 raft_consensus.cc:1226] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Rejecting Update request from peer 606c439025f24e53a2feebeaf68c1681 for earlier term 6. Current term is 7. Ops: [6.14-6.14]
I20240502 07:59:40.532543 3008 raft_consensus.cc:1261] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 5 index: 12. Preceding OpId from leader: term: 7 index: 14. (index mismatch)
I20240502 07:59:40.538483 2614 raft_consensus.cc:2396] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 7 because replica has last-logged OpId of term: 5 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 9.
I20240502 07:59:40.538533 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 11 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.538636 2879 raft_consensus.cc:3032] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.538682 2879 raft_consensus.cc:729] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.538818 2879 consensus_queue.cc:260] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 7.12, Last appended by leader: 12, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.538967 2879 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.539956 2879 raft_consensus.cc:2396] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 8 because replica has last-logged OpId of term: 7 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 11.
I20240502 07:59:40.540306 4910 consensus_queue.cc:1035] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:40.541285 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 7 candidate_status { last_received { term: 6 index: 13 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.541411 2879 raft_consensus.cc:3032] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.541456 2879 raft_consensus.cc:729] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.541435 2946 leader_election.cc:302] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.541585 2879 consensus_queue.cc:260] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 6.14, Last appended by leader: 14, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.541625 3008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.541725 3008 raft_consensus.cc:482] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.541719 2879 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.541772 3008 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.541769 4855 raft_consensus.cc:2781] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.542114 4855 raft_consensus.cc:686] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.542234 4855 consensus_queue.cc:237] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 5.11, Last appended by leader: 11, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.542606 2879 raft_consensus.cc:2396] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 7 because replica has last-logged OpId of term: 6 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 13.
I20240502 07:59:40.542745 3008 raft_consensus.cc:504] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.543102 3008 leader_election.cc:288] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.543654 3008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.543747 3008 raft_consensus.cc:482] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.543792 3008 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.543745 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 7 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.543852 2879 raft_consensus.cc:3032] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.543889 2879 raft_consensus.cc:729] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.544005 2879 consensus_queue.cc:260] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.11, Last appended by leader: 11, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.544131 2879 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.544682 3008 raft_consensus.cc:504] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.544981 2744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 7 candidate_status { last_received { term: 6 index: 13 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.545187 2879 raft_consensus.cc:2396] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 7 because replica has last-logged OpId of term: 6 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 9.
I20240502 07:59:40.545469 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 12 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.546118 2878 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.546236 2744 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.547250 3008 leader_election.cc:288] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.547816 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.547912 3005 raft_consensus.cc:482] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.547957 3005 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.548842 3005 raft_consensus.cc:504] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.549063 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
W20240502 07:59:40.549124 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34834: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.549175 2613 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.549237 2944 leader_election.cc:302] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.549280 3005 leader_election.cc:288] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.549420 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.549502 2614 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.549821 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.549909 3005 raft_consensus.cc:482] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.549957 3005 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.550179 2613 raft_consensus.cc:2396] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 8 because replica has last-logged OpId of term: 6 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 8.
I20240502 07:59:40.550390 2614 raft_consensus.cc:2396] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 8 because replica has last-logged OpId of term: 6 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 11.
I20240502 07:59:40.550769 3005 raft_consensus.cc:504] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.551164 3005 leader_election.cc:288] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 7 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.551676 3008 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.551700 2878 raft_consensus.cc:2454] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 8.
I20240502 07:59:40.551774 3008 raft_consensus.cc:482] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.551828 3008 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.552217 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 26 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.552238 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.552320 2879 raft_consensus.cc:3032] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.552332 2877 raft_consensus.cc:3032] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.552383 2879 raft_consensus.cc:729] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.552405 2877 raft_consensus.cc:729] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.552531 2879 consensus_queue.cc:260] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 7.27, Last appended by leader: 27, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.552542 2877 consensus_queue.cc:260] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.10, Last appended by leader: 10, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.552701 2879 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.552711 2877 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.553576 2879 raft_consensus.cc:2396] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 8 because replica has last-logged OpId of term: 7 index: 27, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 26.
I20240502 07:59:40.553833 2877 raft_consensus.cc:2396] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 8 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 8.
I20240502 07:59:40.554853 3010 raft_consensus.cc:1226] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Rejecting Update request from peer c93a6d22180a41a686c4556afd6a3e6b for earlier term 7. Current term is 8. Ops: [7.13-7.15]
I20240502 07:59:40.555238 4074 consensus_queue.cc:1046] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: INVALID_TERM, Last received: 5.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:40.555390 4074 raft_consensus.cc:3032] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.555438 4074 raft_consensus.cc:729] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.555577 4074 consensus_queue.cc:260] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 7.16, Last appended by leader: 16, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.555747 4074 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.557082 4883 raft_consensus.cc:2726] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 07:59:40.557235 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 12 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.557411 2614 raft_consensus.cc:2396] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 8 because replica has last-logged OpId of term: 7 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 12.
I20240502 07:59:40.557690 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.557792 2878 raft_consensus.cc:3032] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.557837 2878 raft_consensus.cc:729] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.557989 2878 consensus_queue.cc:260] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 7.13, Last appended by leader: 13, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.558135 2878 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.560323 2944 leader_election.cc:302] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters: e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.560611 2944 leader_election.cc:302] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.560765 4885 raft_consensus.cc:2781] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.560842 4885 raft_consensus.cc:686] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.560962 4885 consensus_queue.cc:237] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 5.12, Last appended by leader: 12, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.561411 4855 raft_consensus.cc:2726] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240502 07:59:40.561800 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.561911 2877 raft_consensus.cc:3032] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 6 LEADER]: Stepping down as leader of term 6
I20240502 07:59:40.561954 2877 raft_consensus.cc:729] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.562088 2877 consensus_queue.cc:260] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.562211 2877 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.562394 2745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 12 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.562512 2745 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.563171 2877 raft_consensus.cc:2396] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 07:59:40.563304 2878 raft_consensus.cc:2396] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 8 because replica has last-logged OpId of term: 7 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 11.
I20240502 07:59:40.563493 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.563573 2745 raft_consensus.cc:2396] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 8 because replica has last-logged OpId of term: 7 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 12.
W20240502 07:59:40.563660 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.563584 2747 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.563905 2475 catalog_manager.cc:5484] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 7 to 8, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 8 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.564589 2747 raft_consensus.cc:2396] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 07:59:40.562371 2742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 26 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.565027 2742 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.565929 2742 raft_consensus.cc:2454] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 8.
I20240502 07:59:40.566299 2946 leader_election.cc:302] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.566673 4883 raft_consensus.cc:2726] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240502 07:59:40.567090 2946 leader_election.cc:302] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681
I20240502 07:59:40.567341 4883 raft_consensus.cc:2781] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.567461 4883 raft_consensus.cc:686] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.567839 4883 consensus_queue.cc:237] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 5.26, Last appended by leader: 26, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.568178 2475 catalog_manager.cc:5484] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 6 to 8, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 8 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.567505 2944 leader_election.cc:302] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.568914 2944 leader_election.cc:302] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.569046 4885 raft_consensus.cc:2726] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240502 07:59:40.569108 4885 raft_consensus.cc:2726] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240502 07:59:40.571893 2744 raft_consensus.cc:2454] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 7.
I20240502 07:59:40.572360 2878 raft_consensus.cc:1261] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 6 index: 14. Preceding OpId from leader: term: 7 index: 16. (index mismatch)
I20240502 07:59:40.572731 2946 leader_election.cc:302] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681
I20240502 07:59:40.572772 3008 raft_consensus.cc:504] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.572960 4883 raft_consensus.cc:2781] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.573038 4883 raft_consensus.cc:686] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.573101 3008 leader_election.cc:288] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.573169 4883 consensus_queue.cc:237] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 6.13, Last appended by leader: 13, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.574187 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.574245 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.574350 2878 raft_consensus.cc:3032] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.574353 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "811167eb170b49219b6cd22d865af334"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.574399 2878 raft_consensus.cc:729] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.574451 2614 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.574610 3010 raft_consensus.cc:482] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.574559 2878 consensus_queue.cc:260] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.574720 2878 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.574728 3010 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.575484 2614 raft_consensus.cc:2454] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 8.
I20240502 07:59:40.575708 2878 raft_consensus.cc:2396] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20240502 07:59:40.575820 3010 raft_consensus.cc:504] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.575973 4883 consensus_queue.cc:1035] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 07:59:40.576407 3010 leader_election.cc:288] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.576475 2944 leader_election.cc:302] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.577075 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:40.577209 4883 raft_consensus.cc:2781] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.577231 3010 raft_consensus.cc:482] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.577282 3010 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.577459 4883 raft_consensus.cc:686] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.577595 4883 consensus_queue.cc:237] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.578191 3010 raft_consensus.cc:504] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.578812 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.578933 2614 raft_consensus.cc:3032] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.578876 3010 leader_election.cc:288] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 7 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.578984 2614 raft_consensus.cc:729] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.579138 2614 consensus_queue.cc:260] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.10, Last appended by leader: 10, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.579303 2614 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.579465 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.579552 2613 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.579994 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 7 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.580116 2747 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.580536 2744 raft_consensus.cc:1261] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 6 index: 14. Preceding OpId from leader: term: 7 index: 16. (index mismatch)
I20240502 07:59:40.580549 2613 raft_consensus.cc:2454] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 7.
I20240502 07:59:40.581149 2747 raft_consensus.cc:2396] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 7 because replica has last-logged OpId of term: 5 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 9.
I20240502 07:59:40.581353 4883 consensus_queue.cc:1035] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20240502 07:59:40.581751 2742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 8 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.581907 2742 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.581936 2614 raft_consensus.cc:2396] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 8 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 9.
I20240502 07:59:40.582803 2742 raft_consensus.cc:2454] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 8.
I20240502 07:59:40.583115 2944 leader_election.cc:302] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters: e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.583287 2946 leader_election.cc:302] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.583667 4885 raft_consensus.cc:2781] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.583745 4885 raft_consensus.cc:686] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.583860 4885 consensus_queue.cc:237] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.584043 4713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:40.584255 4855 raft_consensus.cc:2781] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.584328 4855 raft_consensus.cc:686] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.584450 4855 consensus_queue.cc:237] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 4.9, Last appended by leader: 9, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.585076 4806 mvcc.cc:205] Tried to move back new op lower bound from 7023152253217677312 to 7023152252949901312. Current Snapshot: MvccSnapshot[applied={T|T < 7023152253217677312}]
I20240502 07:59:40.588702 2475 catalog_manager.cc:5484] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 7 to 8, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 8 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.593403 2467 catalog_manager.cc:5484] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 7 to 8, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 8 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.593653 2467 catalog_manager.cc:5484] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 5 to 7, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 7 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.593812 2467 catalog_manager.cc:5484] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 6 to 7, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 7 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.593938 2467 catalog_manager.cc:5484] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 7 to 8, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 8 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
W20240502 07:59:40.599511 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45948: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:40.602988 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "512e1322925c415f8891dfe6621160e1"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.603121 2742 raft_consensus.cc:482] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.603171 2742 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
W20240502 07:59:40.603272 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.604162 2742 raft_consensus.cc:504] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.604801 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 9 candidate_status { last_received { term: 8 index: 12 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.604916 3008 raft_consensus.cc:3032] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.604959 3008 raft_consensus.cc:729] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.605080 3008 consensus_queue.cc:260] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 8.12, Last appended by leader: 12, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.605230 3008 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.605777 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 9 candidate_status { last_received { term: 8 index: 12 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.605959 2878 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.606266 3008 raft_consensus.cc:2454] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 9.
I20240502 07:59:40.606528 2681 leader_election.cc:302] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.606868 4906 raft_consensus.cc:2781] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.606945 4906 raft_consensus.cc:686] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 9 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.607055 4906 consensus_queue.cc:237] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 8.12, Last appended by leader: 12, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.607252 2878 raft_consensus.cc:2454] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 9.
I20240502 07:59:40.607610 2742 leader_election.cc:288] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 9 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.609129 2467 catalog_manager.cc:5484] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 8 to 9, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 9 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
W20240502 07:59:40.609248 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.609751 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c67533803b9c496295f4f57e77a51374"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.609899 2742 raft_consensus.cc:482] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.609993 2742 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.611078 2742 raft_consensus.cc:504] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.611456 2742 leader_election.cc:288] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.612205 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.612236 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 6 index: 13 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.612347 2878 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.612376 2742 raft_consensus.cc:482] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.612504 2742 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.613307 2742 raft_consensus.cc:504] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.613976 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 11 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.614073 2877 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
W20240502 07:59:40.614652 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45948: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:40.614979 2877 raft_consensus.cc:2454] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 8.
I20240502 07:59:40.615137 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 6 index: 13 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.615265 3008 raft_consensus.cc:3032] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.615304 3008 raft_consensus.cc:729] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.615496 3008 consensus_queue.cc:260] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 7.14, Last appended by leader: 14, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.615571 2682 leader_election.cc:302] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.615633 3008 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.615726 4193 raft_consensus.cc:2781] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.615788 4193 raft_consensus.cc:686] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.615892 4193 consensus_queue.cc:237] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 5.11, Last appended by leader: 11, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.616493 3008 raft_consensus.cc:2396] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 8 because replica has last-logged OpId of term: 7 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 13.
I20240502 07:59:40.617151 2878 raft_consensus.cc:2396] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 8 because replica has last-logged OpId of term: 6 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 13.
I20240502 07:59:40.617548 2682 leader_election.cc:302] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1; no voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.617575 2742 leader_election.cc:288] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.617861 4906 raft_consensus.cc:2726] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240502 07:59:40.618000 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 11 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.618119 2614 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.618680 2467 catalog_manager.cc:5484] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 6 to 8, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 8 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.618996 2614 raft_consensus.cc:2396] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 8 because replica has last-logged OpId of term: 6 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 11.
W20240502 07:59:40.619860 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.620800 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.620898 2742 raft_consensus.cc:482] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.620941 2742 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.621773 2742 raft_consensus.cc:504] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.622112 2742 leader_election.cc:288] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.622511 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.622622 2878 raft_consensus.cc:3032] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.622668 2878 raft_consensus.cc:729] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.622787 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.622792 2878 consensus_queue.cc:260] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.10, Last appended by leader: 10, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.622895 2614 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.622907 2878 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.623739 2878 raft_consensus.cc:2396] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 8 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 9.
I20240502 07:59:40.623874 2614 raft_consensus.cc:2454] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 8.
I20240502 07:59:40.624142 2682 leader_election.cc:302] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.624406 4906 raft_consensus.cc:2781] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.624518 4906 raft_consensus.cc:686] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.624660 4906 consensus_queue.cc:237] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.624786 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f116334237934021a25a6cacae430034"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.624874 2742 raft_consensus.cc:482] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.624917 2742 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.625728 2742 raft_consensus.cc:504] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.626013 2742 leader_election.cc:288] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.626518 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.626621 3008 raft_consensus.cc:3032] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.626677 3008 raft_consensus.cc:729] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.626791 3008 consensus_queue.cc:260] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.10, Last appended by leader: 10, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.626897 3008 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.627802 3008 raft_consensus.cc:2396] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 8 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 9.
W20240502 07:59:40.628756 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:40.629218 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.629343 2742 raft_consensus.cc:482] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.629261 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.629396 2742 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.629442 2614 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.630153 2742 raft_consensus.cc:504] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.630443 2614 raft_consensus.cc:2454] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 8.
I20240502 07:59:40.630880 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 7 index: 16 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.630970 3008 raft_consensus.cc:3032] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.631002 3008 raft_consensus.cc:729] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.631111 3008 consensus_queue.cc:260] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 7.16, Last appended by leader: 16, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.631063 2467 catalog_manager.cc:5484] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 7 to 8, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 8 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.631239 3008 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.632055 3008 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 8.
I20240502 07:59:40.632342 2681 leader_election.cc:302] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.632493 4193 raft_consensus.cc:2781] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.632551 4193 raft_consensus.cc:686] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.632658 4193 consensus_queue.cc:237] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 7.16, Last appended by leader: 16, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.633625 2682 leader_election.cc:302] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters: f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.633992 4906 raft_consensus.cc:2781] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.634063 4906 raft_consensus.cc:686] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.634156 4906 consensus_queue.cc:237] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.634735 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 7 index: 16 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.634840 2878 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.635671 2878 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 8.
I20240502 07:59:40.636019 2742 leader_election.cc:288] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.636317 2467 catalog_manager.cc:5484] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 7 to 8, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 8 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.636617 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.636566 2467 catalog_manager.cc:5484] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 7 to 8, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 8 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.636704 2747 raft_consensus.cc:482] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.636747 2747 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.637656 2747 raft_consensus.cc:504] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.637953 2747 leader_election.cc:288] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 9 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.638753 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.638842 2747 raft_consensus.cc:482] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.639991 2747 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.646494 2747 raft_consensus.cc:504] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.646838 2747 leader_election.cc:288] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.647404 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 7 index: 11 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.647451 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 7 index: 11 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.647534 2878 raft_consensus.cc:3032] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.647563 2613 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.647578 2878 raft_consensus.cc:729] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.647727 2878 consensus_queue.cc:260] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 7.11, Last appended by leader: 11, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.647882 2878 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.648524 2613 raft_consensus.cc:2454] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 8.
I20240502 07:59:40.648847 2878 raft_consensus.cc:2454] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 8.
I20240502 07:59:40.648890 2682 leader_election.cc:302] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.649047 4193 raft_consensus.cc:2781] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.649143 4193 raft_consensus.cc:686] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.649261 4193 consensus_queue.cc:237] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 7.11, Last appended by leader: 11, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.649690 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 9 candidate_status { last_received { term: 8 index: 23 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.649791 2614 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.650715 2614 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 9.
I20240502 07:59:40.650804 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 9 candidate_status { last_received { term: 8 index: 23 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.650918 3008 raft_consensus.cc:3032] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.650964 3008 raft_consensus.cc:729] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.651146 2682 leader_election.cc:302] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.651111 3008 consensus_queue.cc:260] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 8.23, Last appended by leader: 23, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.651288 3008 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.651458 4193 raft_consensus.cc:2781] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.651530 4193 raft_consensus.cc:686] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 9 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.651643 4193 consensus_queue.cc:237] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 8.23, Last appended by leader: 23, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.651859 2475 catalog_manager.cc:5484] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 7 to 8, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 8 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.652269 3008 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 9.
I20240502 07:59:40.652662 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.657567 2747 raft_consensus.cc:482] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.657696 2747 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.659554 2747 raft_consensus.cc:504] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.659581 2475 catalog_manager.cc:5484] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 8 to 9, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 9 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.659924 2747 leader_election.cc:288] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.660907 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 19 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.661022 2878 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.661947 2878 raft_consensus.cc:2454] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 8.
I20240502 07:59:40.662266 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 19 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.662384 2614 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.663278 2614 raft_consensus.cc:2396] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 8 because replica has last-logged OpId of term: 6 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 19.
I20240502 07:59:40.663704 2682 leader_election.cc:302] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.663854 4906 raft_consensus.cc:2781] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.663935 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.663944 4906 raft_consensus.cc:686] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.664042 2742 raft_consensus.cc:482] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.664086 2742 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.664135 4906 consensus_queue.cc:237] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 5.19, Last appended by leader: 19, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.664942 2742 raft_consensus.cc:504] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.665259 2742 leader_election.cc:288] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.665848 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.665951 2878 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.666241 2475 catalog_manager.cc:5484] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 6 to 8, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 8 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.666855 2878 raft_consensus.cc:2454] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 8.
I20240502 07:59:40.667150 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 10 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.667268 2614 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.668085 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.668193 2614 raft_consensus.cc:2396] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 8 because replica has last-logged OpId of term: 6 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 10.
I20240502 07:59:40.668229 2742 raft_consensus.cc:482] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.668359 2742 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.668536 2682 leader_election.cc:302] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.668676 4906 raft_consensus.cc:2781] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.668756 4906 raft_consensus.cc:686] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.668879 4906 consensus_queue.cc:237] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 5.10, Last appended by leader: 10, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.669196 2742 raft_consensus.cc:504] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.669827 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.669922 2614 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
W20240502 07:59:40.670362 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45948: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:40.670514 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.670789 3008 raft_consensus.cc:3032] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.670847 3008 raft_consensus.cc:729] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.671020 3008 consensus_queue.cc:260] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.10, Last appended by leader: 10, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.671154 3008 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.671320 2614 raft_consensus.cc:2454] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 8.
I20240502 07:59:40.671550 2742 leader_election.cc:288] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.671546 2682 leader_election.cc:302] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.671800 4193 raft_consensus.cc:2781] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.671886 4193 raft_consensus.cc:686] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.672004 4193 consensus_queue.cc:237] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.672160 3008 raft_consensus.cc:2396] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 8 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 9.
I20240502 07:59:40.672384 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.672473 2747 raft_consensus.cc:482] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.672524 2747 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.673249 2747 raft_consensus.cc:504] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.673614 2747 leader_election.cc:288] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 9 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.674201 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 9 candidate_status { last_received { term: 4 index: 25 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.674304 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 9 candidate_status { last_received { term: 4 index: 25 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.674330 2878 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.674391 3008 raft_consensus.cc:3032] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.674427 3008 raft_consensus.cc:729] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.674597 3008 consensus_queue.cc:260] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 8.27, Last appended by leader: 27, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.674710 3008 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.675503 2878 raft_consensus.cc:2396] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 9 because replica has last-logged OpId of term: 7 index: 27, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 25.
I20240502 07:59:40.675503 3008 raft_consensus.cc:2396] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 9 because replica has last-logged OpId of term: 8 index: 27, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 25.
I20240502 07:59:40.675717 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.675829 2742 raft_consensus.cc:482] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.675899 2742 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.676200 2682 leader_election.cc:302] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1; no voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.676589 4906 raft_consensus.cc:2726] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 07:59:40.677379 2742 raft_consensus.cc:504] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.677755 2742 leader_election.cc:288] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 7 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.678058 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 7 candidate_status { last_received { term: 4 index: 24 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.678159 2614 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.678174 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 7 candidate_status { last_received { term: 4 index: 24 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.678265 3008 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 6 FOLLOWER]: Advancing to term 7
I20240502 07:59:40.678431 2467 catalog_manager.cc:5484] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 6 to 8, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 8 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.679075 2614 raft_consensus.cc:2396] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 7 because replica has last-logged OpId of term: 5 index: 24, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 24.
I20240502 07:59:40.679119 3008 raft_consensus.cc:2454] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 7.
I20240502 07:59:40.679605 2681 leader_election.cc:302] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.679623 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.679796 4906 raft_consensus.cc:2781] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Leader election won for term 7
I20240502 07:59:40.679782 2742 raft_consensus.cc:482] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.679885 2742 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.679903 4906 raft_consensus.cc:686] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 7 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.680032 4906 consensus_queue.cc:237] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 4.24, Last appended by leader: 24, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.680761 2742 raft_consensus.cc:504] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
W20240502 07:59:40.681051 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.681142 2742 leader_election.cc:288] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.681660 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.681782 2878 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.681766 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.681881 2614 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.681856 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.681949 2742 raft_consensus.cc:482] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.682003 2742 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.682626 2742 raft_consensus.cc:504] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.682761 2878 raft_consensus.cc:2454] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 8.
I20240502 07:59:40.682878 2742 leader_election.cc:288] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.683169 2614 raft_consensus.cc:2396] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20240502 07:59:40.683279 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.683387 3008 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.684191 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.684314 2878 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.684430 2682 leader_election.cc:302] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.684777 4906 raft_consensus.cc:2781] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.684849 4906 raft_consensus.cc:686] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.684985 4906 consensus_queue.cc:237] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.685338 2878 raft_consensus.cc:2396] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20240502 07:59:40.684424 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.686198 2742 raft_consensus.cc:482] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.686262 2742 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.687094 2742 raft_consensus.cc:504] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.687147 2467 catalog_manager.cc:5484] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 6 to 8, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 8 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.687348 2467 catalog_manager.cc:5484] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 7 to 8, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 8 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.687533 2467 catalog_manager.cc:5484] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 5 to 7, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 7 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.687716 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 9 candidate_status { last_received { term: 7 index: 16 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.687754 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 9 candidate_status { last_received { term: 7 index: 16 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.687811 2614 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.687852 3010 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.688668 2614 raft_consensus.cc:2454] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 9.
I20240502 07:59:40.688715 3010 raft_consensus.cc:2454] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 9.
I20240502 07:59:40.688948 2682 leader_election.cc:302] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.689082 4906 raft_consensus.cc:2781] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.689157 4906 raft_consensus.cc:686] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 9 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.689277 4906 consensus_queue.cc:237] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 7.16, Last appended by leader: 16, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.690338 3008 raft_consensus.cc:2454] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 8.
I20240502 07:59:40.690518 2742 leader_election.cc:288] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 9 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.690670 2681 leader_election.cc:302] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681
I20240502 07:59:40.690816 4906 raft_consensus.cc:2781] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.690944 4906 raft_consensus.cc:686] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.691051 4906 consensus_queue.cc:237] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.691965 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.692090 2742 raft_consensus.cc:482] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.692150 2742 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.693114 2467 catalog_manager.cc:5484] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 7 to 9, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 9 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.693248 2742 raft_consensus.cc:504] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.693485 2467 catalog_manager.cc:5484] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 6 to 8, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 8 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.694003 2742 leader_election.cc:288] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 9 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.694290 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 9 candidate_status { last_received { term: 6 index: 12 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.694413 2878 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.695287 2878 raft_consensus.cc:2454] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 9.
I20240502 07:59:40.695281 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d7b590b4226443d4899f3005840fd736"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.695376 2742 raft_consensus.cc:482] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.695423 2742 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.695621 2682 leader_election.cc:302] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.695755 4193 raft_consensus.cc:2781] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.695814 4193 raft_consensus.cc:686] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 9 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.695915 4193 consensus_queue.cc:237] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 6.12, Last appended by leader: 12, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.696383 2742 raft_consensus.cc:504] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.696832 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 9 candidate_status { last_received { term: 6 index: 12 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.696938 3008 raft_consensus.cc:3032] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.696960 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 9 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.696987 3008 raft_consensus.cc:729] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.697073 3010 raft_consensus.cc:3032] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.697111 3010 raft_consensus.cc:729] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.697142 3008 consensus_queue.cc:260] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.13, Last appended by leader: 13, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.697235 3008 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.697219 3010 consensus_queue.cc:260] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.12, Last appended by leader: 12, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.697583 3010 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.698302 2614 raft_consensus.cc:1261] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 7 index: 16. Preceding OpId from leader: term: 9 index: 18. (index mismatch)
I20240502 07:59:40.698567 3010 raft_consensus.cc:2396] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 9 because replica has last-logged OpId of term: 8 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 10.
I20240502 07:59:40.698933 4906 consensus_queue.cc:1035] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 07:59:40.699700 3010 raft_consensus.cc:1261] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 5 index: 12. Preceding OpId from leader: term: 9 index: 18. (index mismatch)
I20240502 07:59:40.700788 4906 consensus_queue.cc:1035] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 12, Time since last communication: 0.000s
I20240502 07:59:40.701102 2467 catalog_manager.cc:5484] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 8 to 9, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 9 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.703291 3008 raft_consensus.cc:2396] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 9 because replica has last-logged OpId of term: 8 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 12.
W20240502 07:59:40.706913 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.707154 2742 leader_election.cc:288] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 9 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.707567 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 9 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.707684 2878 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.708545 2878 raft_consensus.cc:2396] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 9 because replica has last-logged OpId of term: 7 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 10.
I20240502 07:59:40.708834 2682 leader_election.cc:302] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1; no voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.709074 4193 raft_consensus.cc:2726] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 07:59:40.710366 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.710496 2742 raft_consensus.cc:482] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.710649 2742 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.711565 2742 raft_consensus.cc:504] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.711942 2742 leader_election.cc:288] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 9 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.712266 2878 raft_consensus.cc:1261] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 7 index: 16. Preceding OpId from leader: term: 8 index: 18. (index mismatch)
I20240502 07:59:40.712281 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.712380 2614 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.712694 4193 consensus_queue.cc:1035] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 07:59:40.713290 3010 raft_consensus.cc:1261] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 7 index: 16. Preceding OpId from leader: term: 8 index: 18. (index mismatch)
I20240502 07:59:40.713447 2614 raft_consensus.cc:2396] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20240502 07:59:40.713701 4193 consensus_queue.cc:1035] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20240502 07:59:40.714341 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.714442 2742 raft_consensus.cc:482] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.714494 2742 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.715113 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.715214 3008 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.716202 3008 raft_consensus.cc:2454] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 9.
I20240502 07:59:40.716293 2742 raft_consensus.cc:504] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.716553 2681 leader_election.cc:302] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.716617 2742 leader_election.cc:288] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.716717 4193 raft_consensus.cc:2781] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.716770 4193 raft_consensus.cc:686] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 9 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.716881 4193 consensus_queue.cc:237] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.717425 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.717519 2742 raft_consensus.cc:482] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.717564 2742 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.718343 2742 raft_consensus.cc:504] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.718644 2742 leader_election.cc:288] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.719560 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.719586 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.719682 2613 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.719700 2614 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
W20240502 07:59:40.719889 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:40.720187 2475 catalog_manager.cc:5484] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 7 to 9, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 9 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.720623 2614 raft_consensus.cc:2396] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 8 because replica has last-logged OpId of term: 6 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 17.
I20240502 07:59:40.720646 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 17 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.720722 2613 raft_consensus.cc:2396] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20240502 07:59:40.720742 2877 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.721220 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "811167eb170b49219b6cd22d865af334"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.721343 2742 raft_consensus.cc:482] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.721410 2742 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.721443 2879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.721560 2879 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.722209 2742 raft_consensus.cc:504] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.722395 2879 raft_consensus.cc:2454] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 8.
I20240502 07:59:40.722541 2742 leader_election.cc:288] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 9 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.722733 2682 leader_election.cc:302] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.722918 4906 raft_consensus.cc:2781] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.723009 4906 raft_consensus.cc:686] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.723145 4906 consensus_queue.cc:237] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.723580 2877 raft_consensus.cc:2454] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 8.
I20240502 07:59:40.723783 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 9 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.723876 3010 raft_consensus.cc:3032] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.723920 3010 raft_consensus.cc:729] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.724063 3010 consensus_queue.cc:260] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.724099 2682 leader_election.cc:302] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.724207 3010 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.724289 4906 raft_consensus.cc:2781] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.724357 4906 raft_consensus.cc:686] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.724462 4906 consensus_queue.cc:237] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 5.17, Last appended by leader: 17, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.725152 3010 raft_consensus.cc:2396] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 8.
W20240502 07:59:40.725924 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45948: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:40.726873 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 9 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.726966 2613 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.727635 2613 raft_consensus.cc:2396] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 9 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 8.
I20240502 07:59:40.728586 2681 leader_election.cc:302] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1; no voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.728899 4906 raft_consensus.cc:2726] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 07:59:40.729115 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:40.729215 2742 raft_consensus.cc:482] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.729281 2742 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.729908 2742 raft_consensus.cc:504] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.730221 2742 leader_election.cc:288] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 9 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.730542 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.730655 2878 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.731500 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.731595 3010 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.732295 2466 catalog_manager.cc:5484] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 6 to 8, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 8 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.732569 3010 raft_consensus.cc:2454] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 9.
I20240502 07:59:40.732527 2466 catalog_manager.cc:5484] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 6 to 8, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 8 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.732884 2878 raft_consensus.cc:2396] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 9.
I20240502 07:59:40.732946 2613 raft_consensus.cc:1261] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 8 index: 23. Preceding OpId from leader: term: 9 index: 25. (index mismatch)
I20240502 07:59:40.733283 5097 consensus_queue.cc:1035] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20240502 07:59:40.735177 2681 leader_election.cc:302] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681
I20240502 07:59:40.735478 5097 raft_consensus.cc:2781] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.735554 5097 raft_consensus.cc:686] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 9 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.735668 5097 consensus_queue.cc:237] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.737241 3010 raft_consensus.cc:1261] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 8 index: 23. Preceding OpId from leader: term: 9 index: 25. (index mismatch)
I20240502 07:59:40.737605 4906 consensus_queue.cc:1035] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20240502 07:59:40.741433 2614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "18912e0059054584a990be7ccb4a5771"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.741539 2614 raft_consensus.cc:482] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.741585 2614 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.742543 2614 raft_consensus.cc:504] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.743109 2614 leader_election.cc:288] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 8 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.743080 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 8 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.743201 3008 raft_consensus.cc:3032] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.743240 3008 raft_consensus.cc:729] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.743371 3008 consensus_queue.cc:260] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 7.14, Last appended by leader: 14, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.743508 3008 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.744060 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 8 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.744189 2878 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.744434 3008 raft_consensus.cc:2396] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 8 because replica has last-logged OpId of term: 7 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 07:59:40.744596 2614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f116334237934021a25a6cacae430034"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.744704 2614 raft_consensus.cc:482] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.744760 2614 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.745471 2475 catalog_manager.cc:5484] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 7 to 9, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 9 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.749593 2878 raft_consensus.cc:2396] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 8 because replica has last-logged OpId of term: 6 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 12.
I20240502 07:59:40.749879 2551 leader_election.cc:302] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b; no voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.750123 4910 raft_consensus.cc:2726] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240502 07:59:40.750717 2614 raft_consensus.cc:504] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.752429 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.754868 2742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.754976 2742 raft_consensus.cc:3032] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.755017 2742 raft_consensus.cc:729] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.755259 2742 consensus_queue.cc:260] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.755407 2742 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.756496 2742 raft_consensus.cc:2396] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.756736 3010 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.756842 2614 leader_election.cc:288] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.757704 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.757805 2616 raft_consensus.cc:482] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.757859 2616 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.758842 2616 raft_consensus.cc:504] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.759196 2616 leader_election.cc:288] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.760694 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 11 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.760938 2878 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.761227 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.761339 2616 raft_consensus.cc:482] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.761394 2616 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.761889 2742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 11 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.762208 2616 raft_consensus.cc:504] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.762377 2742 raft_consensus.cc:3032] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.762535 2616 leader_election.cc:288] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.763041 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 11 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.763139 2877 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.764130 2877 raft_consensus.cc:2454] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 9.
I20240502 07:59:40.764573 2551 leader_election.cc:302] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:40.764726 4910 raft_consensus.cc:2781] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.764809 4910 raft_consensus.cc:686] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.764916 4910 consensus_queue.cc:237] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 6.11, Last appended by leader: 11, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.765410 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 11 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.765516 3005 raft_consensus.cc:3032] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.765558 3005 raft_consensus.cc:729] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.765697 3005 consensus_queue.cc:260] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.12, Last appended by leader: 12, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.765843 3005 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.766345 2878 raft_consensus.cc:2454] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 9.
I20240502 07:59:40.766615 2551 leader_election.cc:302] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:40.766767 2742 raft_consensus.cc:729] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.766789 4074 raft_consensus.cc:2781] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.766824 3005 raft_consensus.cc:2396] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 8 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 11.
I20240502 07:59:40.766870 4074 raft_consensus.cc:686] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.766976 4074 consensus_queue.cc:237] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 6.11, Last appended by leader: 11, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.767019 2614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.767112 2614 raft_consensus.cc:482] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.767163 2614 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.767736 2466 catalog_manager.cc:5484] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 8 to 9, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 9 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.768025 2614 raft_consensus.cc:504] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.768455 2742 consensus_queue.cc:260] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.12, Last appended by leader: 12, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.768596 2742 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.769405 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 18 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.769510 3005 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.771379 2473 catalog_manager.cc:5484] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 8 to 9, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 9 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.772702 3010 raft_consensus.cc:2396] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.772836 3005 raft_consensus.cc:2454] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 10.
I20240502 07:59:40.773200 2613 raft_consensus.cc:1226] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Rejecting Update request from peer e0a355821af5483f96aec7e128cc18e1 for earlier term 9. Current term is 10. Ops: [9.19-9.19]
I20240502 07:59:40.773342 2550 leader_election.cc:302] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b; no voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.773298 2742 raft_consensus.cc:2396] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 8 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 11.
I20240502 07:59:40.773545 2550 leader_election.cc:302] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.773772 4074 raft_consensus.cc:2726] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 07:59:40.773800 4910 raft_consensus.cc:2781] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election won for term 10
I20240502 07:59:40.773975 3005 raft_consensus.cc:1226] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Rejecting Update request from peer e0a355821af5483f96aec7e128cc18e1 for earlier term 9. Current term is 10. Ops: [9.19-9.19]
I20240502 07:59:40.773996 4910 raft_consensus.cc:686] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 10 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.773991 4193 consensus_queue.cc:1046] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: INVALID_TERM, Last received: 9.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 07:59:40.774113 4910 consensus_queue.cc:237] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 9.18, Last appended by leader: 18, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.774164 4193 raft_consensus.cc:3032] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:40.774210 4193 raft_consensus.cc:729] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.774334 4193 consensus_queue.cc:260] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 9.19, Last appended by leader: 19, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.774422 5097 consensus_queue.cc:1046] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: INVALID_TERM, Last received: 9.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 07:59:40.774616 4193 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.773721 2614 leader_election.cc:288] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 10 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.775843 2614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.775959 2614 raft_consensus.cc:482] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.776011 2614 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.775929 2473 catalog_manager.cc:5484] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 9 to 10, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 10 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.776917 2614 raft_consensus.cc:504] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.777547 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.777683 2878 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.778788 2878 raft_consensus.cc:2396] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.779134 2742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 18 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.779284 2742 raft_consensus.cc:2396] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 10 because replica has last-logged OpId of term: 9 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 18.
I20240502 07:59:40.779778 2614 leader_election.cc:288] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.780325 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "236a4d787879442aaff6983243f21f85"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.780431 2616 raft_consensus.cc:482] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.780509 2616 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.781517 2616 raft_consensus.cc:504] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.781812 2616 leader_election.cc:288] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.782204 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 3 index: 20 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.782332 3005 raft_consensus.cc:3032] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.782363 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.782399 3005 raft_consensus.cc:729] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.782464 2613 raft_consensus.cc:482] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.782518 2613 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.782559 3005 consensus_queue.cc:260] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 8.22, Last appended by leader: 22, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.782708 3005 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.783409 2613 raft_consensus.cc:504] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.783660 3005 raft_consensus.cc:2396] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 8 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 20.
I20240502 07:59:40.783914 2613 leader_election.cc:288] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 10 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.784117 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.784216 3005 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.784437 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 3 index: 20 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.784549 2878 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.785059 3005 raft_consensus.cc:2454] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 10.
I20240502 07:59:40.785378 2550 leader_election.cc:302] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.785490 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.785543 4074 raft_consensus.cc:2781] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election won for term 10
I20240502 07:59:40.785615 2747 raft_consensus.cc:3032] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:40.785634 4074 raft_consensus.cc:686] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 10 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.785658 2747 raft_consensus.cc:729] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.785759 4074 consensus_queue.cc:237] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.785778 2747 consensus_queue.cc:260] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.785907 2747 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.786886 2747 raft_consensus.cc:2396] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20240502 07:59:40.787173 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.787276 2613 raft_consensus.cc:482] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.787329 2613 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.788058 2613 raft_consensus.cc:504] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.788043 2466 catalog_manager.cc:5484] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 9 to 10, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 10 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.788506 2742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.788630 2742 raft_consensus.cc:3032] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.788679 2742 raft_consensus.cc:729] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.788689 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 11 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.788798 2877 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.788894 2742 consensus_queue.cc:260] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.789043 2742 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.789784 2877 raft_consensus.cc:2454] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 9.
I20240502 07:59:40.789994 2742 raft_consensus.cc:2396] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.790095 2551 leader_election.cc:302] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:40.790206 2878 raft_consensus.cc:2396] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 7 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 20.
I20240502 07:59:40.790256 4910 raft_consensus.cc:2781] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.790326 4910 raft_consensus.cc:686] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.790380 2613 leader_election.cc:288] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.790453 4910 consensus_queue.cc:237] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 7.11, Last appended by leader: 11, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.791014 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.791110 2613 raft_consensus.cc:482] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.791167 2613 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.791302 2551 leader_election.cc:302] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b; no voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.791437 4074 raft_consensus.cc:2726] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 07:59:40.791822 2553 leader_election.cc:302] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b; no voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.792008 4074 raft_consensus.cc:2726] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 07:59:40.792232 2742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 7 index: 11 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.792376 2742 raft_consensus.cc:3032] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.792449 2742 raft_consensus.cc:729] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.792621 2742 consensus_queue.cc:260] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 8.12, Last appended by leader: 12, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.792801 2742 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.792840 2466 catalog_manager.cc:5484] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 8 to 9, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 9 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.793130 2613 raft_consensus.cc:504] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.793710 2742 raft_consensus.cc:2396] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 8 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 11.
I20240502 07:59:40.793684 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 19 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.793814 2747 raft_consensus.cc:3032] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.793854 2747 raft_consensus.cc:729] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.793956 2747 consensus_queue.cc:260] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 8.20, Last appended by leader: 20, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.794067 2747 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.794483 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 19 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.794576 2878 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.795027 2747 raft_consensus.cc:2396] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 8 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 19.
I20240502 07:59:40.796675 2878 raft_consensus.cc:2454] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 9.
I20240502 07:59:40.796926 2551 leader_election.cc:302] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters: e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.797084 4074 raft_consensus.cc:2781] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.797161 4074 raft_consensus.cc:686] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.797215 2613 leader_election.cc:288] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.797283 4074 consensus_queue.cc:237] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 6.19, Last appended by leader: 19, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.798125 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.798249 2613 raft_consensus.cc:482] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.798310 2613 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.799188 2613 raft_consensus.cc:504] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.799646 2613 leader_election.cc:288] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.799926 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 10 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.800036 2747 raft_consensus.cc:3032] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.800091 2747 raft_consensus.cc:729] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.799917 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 10 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.800269 2878 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.800223 2747 consensus_queue.cc:260] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.11, Last appended by leader: 11, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.800287 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.800379 2747 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.800489 2613 raft_consensus.cc:482] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.800628 2613 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.801298 2747 raft_consensus.cc:2396] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 8 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 10.
I20240502 07:59:40.801651 2613 raft_consensus.cc:504] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.802083 2878 raft_consensus.cc:2454] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 9.
I20240502 07:59:40.802065 2473 catalog_manager.cc:5484] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 8 to 9, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 9 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.802320 2551 leader_election.cc:302] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters: e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.802407 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.802526 4074 raft_consensus.cc:2781] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.802536 3005 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.802613 4074 raft_consensus.cc:686] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.802731 4074 consensus_queue.cc:237] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.805238 3005 raft_consensus.cc:2396] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.806216 2473 catalog_manager.cc:5484] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 8 to 9, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 9 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.808642 2613 leader_election.cc:288] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.808967 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.809088 2747 raft_consensus.cc:3032] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.809134 2747 raft_consensus.cc:729] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.809263 2747 consensus_queue.cc:260] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.809787 2747 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.809769 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.809908 2613 raft_consensus.cc:482] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.809988 2613 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.810864 2747 raft_consensus.cc:2396] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.810959 2613 raft_consensus.cc:504] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.811478 2553 leader_election.cc:302] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b; no voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.811674 4910 raft_consensus.cc:2726] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 07:59:40.811991 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 8 candidate_status { last_received { term: 5 index: 24 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.812108 3005 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.812990 3005 raft_consensus.cc:2454] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 8.
I20240502 07:59:40.813293 2550 leader_election.cc:302] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.813510 2613 leader_election.cc:288] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 8 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.814049 4910 raft_consensus.cc:2781] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.814127 4910 raft_consensus.cc:686] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 8 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.814252 4910 consensus_queue.cc:237] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 5.24, Last appended by leader: 24, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.814317 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.814404 2613 raft_consensus.cc:482] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.814450 2613 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.815855 2613 raft_consensus.cc:504] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.816015 2473 catalog_manager.cc:5484] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 7 to 8, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 8 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.816192 4451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:40.816460 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.816576 2878 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.817304 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 8 candidate_status { last_received { term: 5 index: 24 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.817452 2747 raft_consensus.cc:3032] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 7 LEADER]: Stepping down as leader of term 7
I20240502 07:59:40.817495 2747 raft_consensus.cc:729] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.817638 2878 raft_consensus.cc:2454] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 9.
I20240502 07:59:40.817639 2747 consensus_queue.cc:260] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 7.25, Last appended by leader: 25, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.817768 2747 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.817890 2551 leader_election.cc:302] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:40.818034 4074 raft_consensus.cc:2781] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.818094 4074 raft_consensus.cc:686] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.818188 4074 consensus_queue.cc:237] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.818723 2747 raft_consensus.cc:2396] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 8 because replica has last-logged OpId of term: 7 index: 25, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 24.
I20240502 07:59:40.819680 2613 leader_election.cc:288] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.820226 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.820313 2613 raft_consensus.cc:482] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.820358 2613 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.820513 2473 catalog_manager.cc:5484] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 8 to 9, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 9 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.820801 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.820888 2747 raft_consensus.cc:3032] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.820920 2747 raft_consensus.cc:729] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.821063 2747 consensus_queue.cc:260] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.821192 2747 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.821543 2613 raft_consensus.cc:504] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.822224 2747 raft_consensus.cc:2396] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20240502 07:59:40.822243 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 31 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.822373 3005 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.823294 3005 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 10.
I20240502 07:59:40.823871 2550 leader_election.cc:302] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.824031 4910 raft_consensus.cc:2781] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election won for term 10
I20240502 07:59:40.824100 4910 raft_consensus.cc:686] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 10 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.824210 4910 consensus_queue.cc:237] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 9.31, Last appended by leader: 31, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.824946 4584 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:40.825480 2613 leader_election.cc:288] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 10 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.825796 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 10 candidate_status { last_received { term: 9 index: 31 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.825896 2747 raft_consensus.cc:3032] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:40.825942 2747 raft_consensus.cc:729] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.826081 2747 consensus_queue.cc:260] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 9.31, Last appended by leader: 31, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.826237 2747 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.826545 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.826686 2613 raft_consensus.cc:482] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.826779 2613 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.826894 2473 catalog_manager.cc:5484] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 9 to 10, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 10 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.827180 2747 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 10.
I20240502 07:59:40.827679 2613 raft_consensus.cc:504] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.828459 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 10 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.828576 2878 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.829675 2878 raft_consensus.cc:2396] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 7 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 10.
I20240502 07:59:40.830093 2613 leader_election.cc:288] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.830761 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 10 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.830879 3005 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.831763 3005 raft_consensus.cc:2454] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 9.
I20240502 07:59:40.832259 2550 leader_election.cc:302] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681
I20240502 07:59:40.832262 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.832402 2613 raft_consensus.cc:482] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.832448 2613 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.833202 2613 raft_consensus.cc:504] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.833825 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.833920 2878 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.834762 2878 raft_consensus.cc:2396] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 07:59:40.835124 2613 leader_election.cc:288] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.837392 4074 raft_consensus.cc:2781] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.837488 4074 raft_consensus.cc:686] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.837617 4074 consensus_queue.cc:237] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.837893 2859 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:40.839170 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.839275 3005 raft_consensus.cc:3032] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.839309 3005 raft_consensus.cc:729] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:40.839452 3005 consensus_queue.cc:260] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.839589 3005 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.840191 2473 catalog_manager.cc:5484] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 7 to 9, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 9 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.840579 3005 raft_consensus.cc:2396] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 07:59:40.840927 2550 leader_election.cc:302] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b; no voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.841130 4910 raft_consensus.cc:2726] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 07:59:40.841871 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.841974 2613 raft_consensus.cc:482] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.842018 2613 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.843379 2613 raft_consensus.cc:504] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.845432 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 17 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.845634 2878 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.846721 2878 raft_consensus.cc:2454] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 9.
I20240502 07:59:40.846976 2551 leader_election.cc:302] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:40.847131 4910 raft_consensus.cc:2781] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.847189 4910 raft_consensus.cc:686] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.847304 4910 consensus_queue.cc:237] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 6.17, Last appended by leader: 17, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.848546 2613 leader_election.cc:288] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.850384 2473 catalog_manager.cc:5484] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 8 to 9, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 9 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.853499 2613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.853602 2613 raft_consensus.cc:482] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.853662 2613 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.854564 2613 raft_consensus.cc:504] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.855307 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.855412 2878 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.856279 2878 raft_consensus.cc:2454] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 9.
I20240502 07:59:40.856539 2551 leader_election.cc:302] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:40.856726 4074 raft_consensus.cc:2781] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.856806 4074 raft_consensus.cc:686] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.856926 4074 consensus_queue.cc:237] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.857911 2597 tablet_service.cc:1429] Tablet server has 13 leaders and 0 scanners
I20240502 07:59:40.857908 2742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.858029 2742 raft_consensus.cc:3032] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.858073 2742 raft_consensus.cc:729] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.858112 2613 leader_election.cc:288] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.858233 2742 consensus_queue.cc:260] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.858383 2742 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.858662 2616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.858753 2616 raft_consensus.cc:482] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.858799 2616 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.859347 2742 raft_consensus.cc:2396] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20240502 07:59:40.859654 2616 raft_consensus.cc:504] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.859964 2616 leader_election.cc:288] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 8 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.860569 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 8 candidate_status { last_received { term: 5 index: 11 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.860705 3005 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.860792 2614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.860894 2614 raft_consensus.cc:482] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.860956 2614 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.861789 3005 raft_consensus.cc:2454] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 8.
I20240502 07:59:40.861788 2614 raft_consensus.cc:504] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.861995 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 17 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.862113 2614 leader_election.cc:288] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:40.862141 2747 raft_consensus.cc:3032] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.862262 2747 raft_consensus.cc:729] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.862457 2747 consensus_queue.cc:260] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 8.18, Last appended by leader: 18, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.862645 2747 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.862717 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 13 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.862846 2877 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.862872 2614 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "811167eb170b49219b6cd22d865af334"
dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
from {username='slave'} at 127.0.0.1:45958
I20240502 07:59:40.862972 2614 raft_consensus.cc:482] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.863039 2614 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.863812 2614 raft_consensus.cc:504] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.863864 2877 raft_consensus.cc:2396] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 7 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 13.
I20240502 07:59:40.863869 2747 raft_consensus.cc:2396] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 9 because replica has last-logged OpId of term: 8 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 17.
I20240502 07:59:40.864501 2614 leader_election.cc:288] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 10 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.864748 2550 leader_election.cc:302] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.864861 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 8 candidate_status { last_received { term: 5 index: 11 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:40.864915 4910 raft_consensus.cc:2781] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Leader election won for term 8
I20240502 07:59:40.864969 2878 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 7 FOLLOWER]: Advancing to term 8
I20240502 07:59:40.864982 4910 raft_consensus.cc:686] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 8 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.865108 4910 consensus_queue.cc:237] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 5.11, Last appended by leader: 11, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.865943 2878 raft_consensus.cc:2396] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 8 because replica has last-logged OpId of term: 6 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 11.
I20240502 07:59:40.866171 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 10 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.866286 3010 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.866442 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 9 candidate_status { last_received { term: 6 index: 13 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.866542 3005 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.867265 3010 raft_consensus.cc:2396] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate c93a6d22180a41a686c4556afd6a3e6b for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 10.
I20240502 07:59:40.867297 3005 raft_consensus.cc:2454] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 9.
I20240502 07:59:40.867636 2550 leader_election.cc:302] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681
I20240502 07:59:40.867820 4074 raft_consensus.cc:2781] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.867884 4074 raft_consensus.cc:686] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.867977 4074 consensus_queue.cc:237] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 6.13, Last appended by leader: 13, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.869364 2473 catalog_manager.cc:5484] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 8 to 9, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 9 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.869625 2473 catalog_manager.cc:5484] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 6 to 8, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 8 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.872473 2727 tablet_service.cc:1429] Tablet server has 5 leaders and 0 scanners
I20240502 07:59:40.878302 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "c93a6d22180a41a686c4556afd6a3e6b" candidate_term: 10 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.878430 2747 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.879410 2747 raft_consensus.cc:2454] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c93a6d22180a41a686c4556afd6a3e6b in term 10.
I20240502 07:59:40.879709 2553 leader_election.cc:302] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters: f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.879945 4910 raft_consensus.cc:2781] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election won for term 10
I20240502 07:59:40.880028 4910 raft_consensus.cc:686] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 10 LEADER]: Becoming Leader. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.880175 4910 consensus_queue.cc:237] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.881227 4320 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:40.882112 2473 catalog_manager.cc:5484] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 7 to 9, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 9 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.886606 2473 catalog_manager.cc:5484] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b reported cstate change: term changed from 8 to 10, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129). New cstate: current_term: 10 leader_uuid: "c93a6d22180a41a686c4556afd6a3e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.889231 4713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:40.893100 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "512e1322925c415f8891dfe6621160e1"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.893231 2878 raft_consensus.cc:482] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.893285 2878 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.893684 2989 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:40.894330 2878 raft_consensus.cc:504] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.894713 2878 leader_election.cc:288] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.895028 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 8 index: 12 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.895155 3010 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.895758 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 8 index: 12 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.895870 2747 raft_consensus.cc:3032] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:40.895905 2747 raft_consensus.cc:729] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.896034 2747 consensus_queue.cc:260] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 9.13, Last appended by leader: 13, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.896111 3010 raft_consensus.cc:2454] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 10.
I20240502 07:59:40.896150 2747 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.896421 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.896427 2812 leader_election.cc:302] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.896534 2878 raft_consensus.cc:482] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.896584 2878 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.897094 2747 raft_consensus.cc:2396] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 9 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 12.
I20240502 07:59:40.897212 2878 raft_consensus.cc:504] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.897289 4993 raft_consensus.cc:2781] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election won for term 10
I20240502 07:59:40.897413 4993 raft_consensus.cc:686] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.897526 4993 consensus_queue.cc:237] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 8.12, Last appended by leader: 12, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.897872 2878 leader_election.cc:288] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.899490 2473 catalog_manager.cc:5484] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 9 to 10, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 10 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.902724 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.902853 2613 raft_consensus.cc:3032] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:40.902896 2613 raft_consensus.cc:729] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.903061 2613 consensus_queue.cc:260] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.12, Last appended by leader: 12, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.903200 2613 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.904163 2613 raft_consensus.cc:2396] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 9 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 10.
I20240502 07:59:40.906160 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.906211 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.906294 2747 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.906329 2878 raft_consensus.cc:482] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.906387 2878 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.907267 2878 raft_consensus.cc:504] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.907372 2747 raft_consensus.cc:2396] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 8 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 10.
I20240502 07:59:40.907615 2878 leader_election.cc:288] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.908250 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.908274 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "18912e0059054584a990be7ccb4a5771"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.908370 2878 raft_consensus.cc:482] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.908429 2878 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.908457 2613 raft_consensus.cc:3032] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:40.908569 2613 raft_consensus.cc:729] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.908753 2613 consensus_queue.cc:260] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.12, Last appended by leader: 12, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.908932 2613 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.909289 2878 raft_consensus.cc:504] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.909662 2878 leader_election.cc:288] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 9 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.909969 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 4 index: 10 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.910090 3010 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.910228 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 9 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.910333 2614 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.910549 2613 raft_consensus.cc:2396] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 9 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 10.
I20240502 07:59:40.910859 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.910948 2878 raft_consensus.cc:482] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.910991 2878 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.911150 3010 raft_consensus.cc:2396] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 8 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 10.
I20240502 07:59:40.911227 2614 raft_consensus.cc:2454] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 9.
I20240502 07:59:40.911381 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 9 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.911466 3005 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.911499 2813 leader_election.cc:302] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:40.911659 4525 raft_consensus.cc:2781] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.911731 4525 raft_consensus.cc:686] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 9 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.911834 4525 consensus_queue.cc:237] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 6.14, Last appended by leader: 14, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.912196 3005 raft_consensus.cc:2396] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 9 because replica has last-logged OpId of term: 7 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 14.
I20240502 07:59:40.912179 2878 raft_consensus.cc:504] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.912719 2878 leader_election.cc:288] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 9 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.912869 2812 leader_election.cc:302] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.912998 4993 raft_consensus.cc:2726] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240502 07:59:40.913386 2815 leader_election.cc:302] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.913523 4993 raft_consensus.cc:2726] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240502 07:59:40.913652 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.913705 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 9 candidate_status { last_received { term: 8 index: 18 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.913740 2878 raft_consensus.cc:482] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.913790 2878 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.913811 3005 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.914528 3005 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 9.
I20240502 07:59:40.914537 2878 raft_consensus.cc:504] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.914809 2812 leader_election.cc:302] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.915071 2878 leader_election.cc:288] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.915138 2742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 7 index: 11 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.915254 2742 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.915192 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 7 index: 11 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.915482 2613 raft_consensus.cc:3032] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:40.915589 2613 raft_consensus.cc:729] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.915932 2613 consensus_queue.cc:260] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.12, Last appended by leader: 12, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.916062 2473 catalog_manager.cc:5484] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 7 to 9, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 9 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:40.916146 2613 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.916250 2742 raft_consensus.cc:2396] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 8 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 11.
I20240502 07:59:40.916894 4993 raft_consensus.cc:2781] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.916993 4993 raft_consensus.cc:686] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 9 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.917075 2613 raft_consensus.cc:2396] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 9 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 11.
I20240502 07:59:40.917122 4993 consensus_queue.cc:237] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 8.18, Last appended by leader: 18, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.917344 2813 leader_election.cc:302] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.917506 4990 raft_consensus.cc:2726] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240502 07:59:40.917928 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.918025 2878 raft_consensus.cc:482] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.918074 2878 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.918949 2878 raft_consensus.cc:504] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.919500 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.919595 2613 raft_consensus.cc:3032] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:40.919642 2613 raft_consensus.cc:729] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.919847 2613 consensus_queue.cc:260] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 9.20, Last appended by leader: 20, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.919984 2613 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.920519 2878 leader_election.cc:288] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.920586 2742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.920701 2742 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.921499 2613 raft_consensus.cc:2396] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 9 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 18.
I20240502 07:59:40.921598 2742 raft_consensus.cc:2396] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 8 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 18.
I20240502 07:59:40.921903 2466 catalog_manager.cc:5484] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 8 to 9, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 9 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.922029 2815 leader_election.cc:302] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.922204 4525 raft_consensus.cc:2726] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240502 07:59:40.922668 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 9 candidate_status { last_received { term: 8 index: 18 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.922770 2747 raft_consensus.cc:3032] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.922804 2747 raft_consensus.cc:729] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.922937 2747 consensus_queue.cc:260] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 8.18, Last appended by leader: 18, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.923094 2747 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.923923 2747 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 9.
I20240502 07:59:40.928371 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.928478 2878 raft_consensus.cc:482] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.928524 2878 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.929311 2878 raft_consensus.cc:504] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.929997 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.930004 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.930092 2747 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.930189 2613 raft_consensus.cc:3032] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:40.930264 2613 raft_consensus.cc:729] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.930444 2613 consensus_queue.cc:260] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.930639 2613 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.930987 2747 raft_consensus.cc:2396] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 8 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 9.
I20240502 07:59:40.931275 2878 leader_election.cc:288] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.931805 2613 raft_consensus.cc:2396] T 4648d978bccc4a65b30342ceaa7cf880 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 9 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 9.
I20240502 07:59:40.931829 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.931931 2878 raft_consensus.cc:482] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.931982 2878 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.932232 2813 leader_election.cc:302] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.932423 4525 raft_consensus.cc:2726] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240502 07:59:40.932811 2878 raft_consensus.cc:504] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.933475 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.933593 3005 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.934415 3005 raft_consensus.cc:2454] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 9.
I20240502 07:59:40.934674 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.934687 2812 leader_election.cc:302] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.934769 2747 raft_consensus.cc:3032] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.934808 2747 raft_consensus.cc:729] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.934845 4525 raft_consensus.cc:2781] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.934928 4525 raft_consensus.cc:686] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 9 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.934934 2747 consensus_queue.cc:260] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.935053 2747 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.935042 4525 consensus_queue.cc:237] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.935860 2747 raft_consensus.cc:2396] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20240502 07:59:40.936584 2466 catalog_manager.cc:5484] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 8 to 9, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 9 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.941388 2878 leader_election.cc:288] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 9 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.942060 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.942169 2878 raft_consensus.cc:482] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.942270 2878 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.943095 2878 raft_consensus.cc:504] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.943413 2878 leader_election.cc:288] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.943671 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.943764 2747 raft_consensus.cc:3032] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:40.943797 2747 raft_consensus.cc:729] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.943912 2747 consensus_queue.cc:260] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.944022 2747 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.944903 2747 raft_consensus.cc:2396] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20240502 07:59:40.949973 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.950105 3005 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.951040 3005 raft_consensus.cc:2454] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 10.
I20240502 07:59:40.951341 2812 leader_election.cc:302] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters: e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.951653 4525 raft_consensus.cc:2781] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election won for term 10
I20240502 07:59:40.951676 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.951728 4525 raft_consensus.cc:686] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.951783 2878 raft_consensus.cc:482] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.951839 2878 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.951843 4525 consensus_queue.cc:237] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.952801 2878 raft_consensus.cc:504] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.953090 2878 leader_election.cc:288] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.953802 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 7 index: 27 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.953904 3005 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.954082 2466 catalog_manager.cc:5484] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 9 to 10, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 10 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.954749 3005 raft_consensus.cc:2396] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 8 index: 27, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 27.
I20240502 07:59:40.955262 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 7 index: 27 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.955387 2747 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.956259 2747 raft_consensus.cc:2454] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 10.
I20240502 07:59:40.956559 2815 leader_election.cc:302] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.956821 4525 raft_consensus.cc:2781] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election won for term 10
I20240502 07:59:40.956898 4525 raft_consensus.cc:686] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.956959 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.956996 4525 consensus_queue.cc:237] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 7.27, Last appended by leader: 27, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.957046 2878 raft_consensus.cc:482] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.957084 2878 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.958001 2878 raft_consensus.cc:504] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.958348 2878 leader_election.cc:288] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.958640 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.958750 2613 raft_consensus.cc:3032] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:40.958786 2613 raft_consensus.cc:729] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.958802 2466 catalog_manager.cc:5484] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 8 to 10, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 10 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.958914 2613 consensus_queue.cc:260] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.959024 2613 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.959969 2613 raft_consensus.cc:2396] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 9 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 10.
I20240502 07:59:40.960553 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.960660 3005 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.961501 3005 raft_consensus.cc:2454] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 10.
I20240502 07:59:40.961920 2812 leader_election.cc:302] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.962138 4525 raft_consensus.cc:2781] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election won for term 10
I20240502 07:59:40.962170 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.962277 2878 raft_consensus.cc:482] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.962335 2878 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.962204 4525 raft_consensus.cc:686] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.962755 4525 consensus_queue.cc:237] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.964370 2466 catalog_manager.cc:5484] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 9 to 10, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 10 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.964885 2878 raft_consensus.cc:504] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.965484 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.965595 2747 raft_consensus.cc:3032] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:40.965651 2747 raft_consensus.cc:729] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:40.965798 2747 consensus_queue.cc:260] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.13, Last appended by leader: 13, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.965925 2747 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.966904 2747 raft_consensus.cc:2396] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 9 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.967722 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.967828 3005 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.968501 2878 leader_election.cc:288] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.968796 3005 raft_consensus.cc:2396] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 8 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 8.
I20240502 07:59:40.969213 2812 leader_election.cc:302] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:40.969403 4525 raft_consensus.cc:2726] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240502 07:59:40.969542 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d7b590b4226443d4899f3005840fd736"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.969628 2878 raft_consensus.cc:482] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.969686 2878 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.970275 2878 raft_consensus.cc:504] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.970573 2878 leader_election.cc:288] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:40.970890 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 7 index: 12 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.970983 2747 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.971624 2747 raft_consensus.cc:2454] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 10.
I20240502 07:59:40.971879 2815 leader_election.cc:302] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.972033 4525 raft_consensus.cc:2781] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election won for term 10
I20240502 07:59:40.972111 4525 raft_consensus.cc:686] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.972234 4525 consensus_queue.cc:237] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 7.12, Last appended by leader: 12, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:40.973389 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 7 index: 12 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.973516 3005 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.973688 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c67533803b9c496295f4f57e77a51374"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.973829 2878 raft_consensus.cc:482] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.973923 2878 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.974504 3005 raft_consensus.cc:2396] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 8 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 12.
I20240502 07:59:40.974941 2878 raft_consensus.cc:504] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.975236 2878 leader_election.cc:288] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 9 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.975533 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 9 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.975646 2747 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.975986 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 9 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.976078 3005 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.976570 2747 raft_consensus.cc:2454] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 9.
I20240502 07:59:40.977068 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.977164 2878 raft_consensus.cc:482] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.977222 2878 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.977304 2815 leader_election.cc:302] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:40.977725 3005 raft_consensus.cc:2396] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 9 because replica has last-logged OpId of term: 7 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 14.
I20240502 07:59:40.978132 4525 raft_consensus.cc:2781] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.978111 2878 raft_consensus.cc:504] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.978211 4525 raft_consensus.cc:686] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 9 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.978320 4525 consensus_queue.cc:237] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 6.14, Last appended by leader: 14, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.978441 2878 leader_election.cc:288] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 9 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.978771 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 9 candidate_status { last_received { term: 6 index: 11 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.978895 2613 raft_consensus.cc:3032] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:40.978943 2613 raft_consensus.cc:729] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.978994 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 9 candidate_status { last_received { term: 6 index: 11 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.979077 3010 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.979092 2613 consensus_queue.cc:260] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.12, Last appended by leader: 12, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.979245 2613 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:40.979200 2466 catalog_manager.cc:5484] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 8 to 10, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 10 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:40.979964 3010 raft_consensus.cc:2454] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 9.
I20240502 07:59:40.980252 2613 raft_consensus.cc:2396] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 9 because replica has last-logged OpId of term: 8 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 11.
I20240502 07:59:40.980443 2812 leader_election.cc:302] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:40.980634 4993 raft_consensus.cc:2781] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Leader election won for term 9
I20240502 07:59:40.980715 4993 raft_consensus.cc:686] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 9 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.980823 4993 consensus_queue.cc:237] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.11, Last appended by leader: 11, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.982065 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.982185 2878 raft_consensus.cc:482] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.982246 2878 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.983191 2878 raft_consensus.cc:504] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.983527 2878 leader_election.cc:288] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.983845 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 7 index: 13 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.983879 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 7 index: 13 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.983954 3005 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.983981 2613 raft_consensus.cc:3032] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:40.984022 2613 raft_consensus.cc:729] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.984272 2613 consensus_queue.cc:260] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.14, Last appended by leader: 14, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.984424 2613 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.985203 2613 raft_consensus.cc:2396] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 9 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 13.
I20240502 07:59:40.985221 3005 raft_consensus.cc:2454] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 10.
I20240502 07:59:40.985558 2812 leader_election.cc:302] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:40.985546 2466 catalog_manager.cc:5484] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 7 to 9, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 9 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.985746 4525 raft_consensus.cc:2781] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election won for term 10
I20240502 07:59:40.985970 4525 raft_consensus.cc:686] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.985987 2466 catalog_manager.cc:5484] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 8 to 9, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 9 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.986155 4525 consensus_queue.cc:237] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 7.13, Last appended by leader: 13, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.986562 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.986908 2878 raft_consensus.cc:482] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.986965 2878 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.988233 2878 raft_consensus.cc:504] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.988560 2878 leader_election.cc:288] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.988934 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.989032 2613 raft_consensus.cc:3032] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:40.989059 2613 raft_consensus.cc:729] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.989192 2613 consensus_queue.cc:260] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.989291 2613 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.990048 2613 raft_consensus.cc:2396] T 6dc9c49ec77547468d9d91c1bbee72d1 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 07:59:40.990012 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dc9c49ec77547468d9d91c1bbee72d1" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.990125 2747 raft_consensus.cc:3037] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.991108 2747 raft_consensus.cc:2396] T 6dc9c49ec77547468d9d91c1bbee72d1 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 07:59:40.991403 2815 leader_election.cc:302] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.991957 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.992059 2878 raft_consensus.cc:482] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.992110 2878 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.992201 4525 raft_consensus.cc:2726] T 6dc9c49ec77547468d9d91c1bbee72d1 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240502 07:59:40.992812 2878 raft_consensus.cc:504] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.993091 2878 leader_election.cc:288] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:40.993402 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.993511 2613 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.993572 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:40.993705 3005 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.993788 2466 catalog_manager.cc:5484] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 9 to 10, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 10 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:40.994421 2613 raft_consensus.cc:2454] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 10.
I20240502 07:59:40.994632 3005 raft_consensus.cc:2396] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20240502 07:59:40.994751 2813 leader_election.cc:302] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:40.994902 4525 raft_consensus.cc:2781] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election won for term 10
I20240502 07:59:40.995054 4525 raft_consensus.cc:686] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:40.995079 2878 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.995285 2878 raft_consensus.cc:482] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.995359 2878 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.995193 4525 consensus_queue.cc:237] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:40.996215 2878 raft_consensus.cc:504] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.996819 2878 leader_election.cc:288] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.996995 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 2 index: 15 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.997103 2613 raft_consensus.cc:3032] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:40.997152 2613 raft_consensus.cc:729] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:40.997280 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9b6915141b94202a356e9e6ae8b71ba" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 2 index: 15 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.997305 2613 consensus_queue.cc:260] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 9.18, Last appended by leader: 18, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:40.997437 2747 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.997526 2613 raft_consensus.cc:3037] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.997546 2877 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:40.997661 2877 raft_consensus.cc:482] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:40.997721 2877 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.998497 2613 raft_consensus.cc:2396] T b9b6915141b94202a356e9e6ae8b71ba P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 9 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 15.
I20240502 07:59:40.998522 2747 raft_consensus.cc:2396] T b9b6915141b94202a356e9e6ae8b71ba P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 8 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 15.
I20240502 07:59:40.998569 2877 raft_consensus.cc:504] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:40.998832 2815 leader_election.cc:302] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:40.998893 2877 leader_election.cc:288] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:40.999285 4525 raft_consensus.cc:2726] T b9b6915141b94202a356e9e6ae8b71ba P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240502 07:59:40.999619 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:40.999716 2747 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:40.999835 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb730198ac6b40c094863fbd7a57669a" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 2 index: 6 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:40.999944 2613 raft_consensus.cc:3032] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:40.999887 2466 catalog_manager.cc:5484] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 8 to 10, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 10 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:40.999994 2613 raft_consensus.cc:729] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 9 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:41.000150 2613 consensus_queue.cc:260] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:41.000310 2613 raft_consensus.cc:3037] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.000605 2877 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "236a4d787879442aaff6983243f21f85"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:41.000710 2877 raft_consensus.cc:482] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.000738 2747 raft_consensus.cc:2396] T eb730198ac6b40c094863fbd7a57669a P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 07:59:41.000761 2877 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.001214 2613 raft_consensus.cc:2396] T eb730198ac6b40c094863fbd7a57669a P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 6.
I20240502 07:59:41.001543 2813 leader_election.cc:302] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:41.001674 2877 raft_consensus.cc:504] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.001700 4525 raft_consensus.cc:2726] T eb730198ac6b40c094863fbd7a57669a P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240502 07:59:41.002000 2877 leader_election.cc:288] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:41.002256 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 7 index: 22 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.002413 2613 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.002530 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 7 index: 22 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:41.002661 3005 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.002897 2877 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d"
dest_uuid: "606c439025f24e53a2feebeaf68c1681"
from {username='slave'} at 127.0.0.1:34780
I20240502 07:59:41.003005 2877 raft_consensus.cc:482] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.003064 2877 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.003264 2613 raft_consensus.cc:2454] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 10.
I20240502 07:59:41.003535 2813 leader_election.cc:302] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:41.003695 4525 raft_consensus.cc:2781] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election won for term 10
I20240502 07:59:41.003788 4525 raft_consensus.cc:686] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:41.003885 2877 raft_consensus.cc:504] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.003913 3005 raft_consensus.cc:2396] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 8 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 22.
I20240502 07:59:41.003926 4525 consensus_queue.cc:237] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 7.22, Last appended by leader: 22, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.004570 2877 leader_election.cc:288] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:41.004742 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.004771 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "606c439025f24e53a2feebeaf68c1681" candidate_term: 10 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:41.004833 2613 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.004854 2747 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.005641 2466 catalog_manager.cc:5484] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 8 to 10, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 10 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:41.005784 2747 raft_consensus.cc:2396] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 606c439025f24e53a2feebeaf68c1681 for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 10.
I20240502 07:59:41.005810 2613 raft_consensus.cc:2454] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 606c439025f24e53a2feebeaf68c1681 in term 10.
I20240502 07:59:41.006043 2813 leader_election.cc:302] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b; no voters:
I20240502 07:59:41.006242 4993 raft_consensus.cc:2781] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election won for term 10
I20240502 07:59:41.006323 4993 raft_consensus.cc:686] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Becoming Leader. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:41.006451 4993 consensus_queue.cc:237] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.008854 2466 catalog_manager.cc:5484] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 reported cstate change: term changed from 8 to 10, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to 606c439025f24e53a2feebeaf68c1681 (127.15.129.131). New cstate: current_term: 10 leader_uuid: "606c439025f24e53a2feebeaf68c1681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:41.024868 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "512e1322925c415f8891dfe6621160e1"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.025046 3005 raft_consensus.cc:482] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.025125 3005 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.026211 3005 raft_consensus.cc:504] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.026620 3005 leader_election.cc:288] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:41.026979 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 8 index: 12 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:41.026983 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 8 index: 12 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.027089 2747 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.027127 2878 raft_consensus.cc:3032] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Stepping down as leader of term 10
I20240502 07:59:41.027172 2878 raft_consensus.cc:729] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:41.027325 2878 consensus_queue.cc:260] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 10.13, Last appended by leader: 13, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.027482 2878 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.028026 2747 raft_consensus.cc:2396] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 11 because replica has last-logged OpId of term: 9 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 12.
I20240502 07:59:41.028517 2878 raft_consensus.cc:2396] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 11 because replica has last-logged OpId of term: 10 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 12.
I20240502 07:59:41.028901 2944 leader_election.cc:302] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:41.029155 4883 raft_consensus.cc:2726] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20240502 07:59:41.031270 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.031405 3005 raft_consensus.cc:482] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.031474 3005 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.034399 3005 raft_consensus.cc:504] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:41.035125 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 8 index: 12 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.035243 2878 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.035270 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7491c293498f4f93a23c62862dce44e5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 8 index: 12 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.035395 2613 raft_consensus.cc:3037] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.036041 2878 raft_consensus.cc:2454] T 7491c293498f4f93a23c62862dce44e5 P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 11.
I20240502 07:59:41.036422 2613 raft_consensus.cc:2396] T 7491c293498f4f93a23c62862dce44e5 P c93a6d22180a41a686c4556afd6a3e6b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 11 because replica has last-logged OpId of term: 9 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 12.
I20240502 07:59:41.036351 2944 leader_election.cc:302] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:41.040768 4883 raft_consensus.cc:2781] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Leader election won for term 11
I20240502 07:59:41.040858 4883 raft_consensus.cc:686] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:41.041004 4883 consensus_queue.cc:237] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.12, Last appended by leader: 12, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:41.034790 3005 leader_election.cc:288] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:41.043004 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.043139 3005 raft_consensus.cc:482] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.043259 3005 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.043331 2468 catalog_manager.cc:5484] T 7491c293498f4f93a23c62862dce44e5 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 9 to 11, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 11 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:41.047147 3005 raft_consensus.cc:504] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.047888 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 9 index: 31 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.047919 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 9 index: 31 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:41.048017 2613 raft_consensus.cc:3032] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 10 LEADER]: Stepping down as leader of term 10
I20240502 07:59:41.048043 2747 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.048069 2613 raft_consensus.cc:729] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:41.048224 2613 consensus_queue.cc:260] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 10.32, Last appended by leader: 32, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.048404 2613 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.048961 2747 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 11.
I20240502 07:59:41.049170 2613 raft_consensus.cc:2396] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 11 because replica has last-logged OpId of term: 10 index: 32, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 31.
I20240502 07:59:41.049250 2946 leader_election.cc:302] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:41.049449 4853 raft_consensus.cc:2781] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Leader election won for term 11
I20240502 07:59:41.049597 4853 raft_consensus.cc:686] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:41.049715 4853 consensus_queue.cc:237] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 9.31, Last appended by leader: 31, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.047590 3005 leader_election.cc:288] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:41.051306 2470 catalog_manager.cc:5484] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 10 to 11, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 11 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:41.052163 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "236a4d787879442aaff6983243f21f85"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.052276 3005 raft_consensus.cc:482] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.052340 3005 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.053164 3005 raft_consensus.cc:504] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.053547 3005 leader_election.cc:288] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:41.054054 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 8 index: 22 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.054162 2613 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.054440 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.054569 3005 raft_consensus.cc:482] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.054672 3005 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.055152 2613 raft_consensus.cc:2454] T 236a4d787879442aaff6983243f21f85 P c93a6d22180a41a686c4556afd6a3e6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 11.
I20240502 07:59:41.055625 3005 raft_consensus.cc:504] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:41.056052 2944 leader_election.cc:302] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:41.056260 4853 raft_consensus.cc:2781] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Leader election won for term 11
I20240502 07:59:41.056270 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:41.056349 4853 raft_consensus.cc:686] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:41.056406 2747 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.056479 4853 consensus_queue.cc:237] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 8.22, Last appended by leader: 22, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.057382 2747 raft_consensus.cc:2396] T 6585f79fc41a4418b124a3e1331699c2 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 07:59:41.057806 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "236a4d787879442aaff6983243f21f85" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 8 index: 22 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.057932 2878 raft_consensus.cc:3032] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Stepping down as leader of term 10
I20240502 07:59:41.057979 2878 raft_consensus.cc:729] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:41.058125 2878 consensus_queue.cc:260] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 10.23, Last appended by leader: 23, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.058254 2878 raft_consensus.cc:3037] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.058596 2470 catalog_manager.cc:5484] T 236a4d787879442aaff6983243f21f85 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 10 to 11, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 11 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:41.059237 2878 raft_consensus.cc:2396] T 236a4d787879442aaff6983243f21f85 P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 11 because replica has last-logged OpId of term: 10 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 22.
I20240502 07:59:41.059937 3005 leader_election.cc:288] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:41.060606 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6585f79fc41a4418b124a3e1331699c2" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.060725 2613 raft_consensus.cc:3032] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 10 LEADER]: Stepping down as leader of term 10
I20240502 07:59:41.060773 2613 raft_consensus.cc:729] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:41.060910 2613 consensus_queue.cc:260] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:41.061053 2613 raft_consensus.cc:3037] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.061869 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.061985 3005 raft_consensus.cc:482] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.062064 3005 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:41.062099 2613 raft_consensus.cc:2396] T 6585f79fc41a4418b124a3e1331699c2 P c93a6d22180a41a686c4556afd6a3e6b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20240502 07:59:41.062933 2944 leader_election.cc:302] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:41.063055 3005 raft_consensus.cc:504] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.063187 4853 raft_consensus.cc:2726] T 6585f79fc41a4418b124a3e1331699c2 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20240502 07:59:41.063421 3005 leader_election.cc:288] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 9 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:41.063673 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 9 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.063766 2613 raft_consensus.cc:3032] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 8 LEADER]: Stepping down as leader of term 8
I20240502 07:59:41.063815 2613 raft_consensus.cc:729] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:41.063931 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6518d4af37a1490b88f86b3de2677a5a" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 9 candidate_status { last_received { term: 3 index: 23 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:41.064034 2747 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:41.063977 2613 consensus_queue.cc:260] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 8.25, Last appended by leader: 25, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.064124 2613 raft_consensus.cc:3037] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 8 FOLLOWER]: Advancing to term 9
I20240502 07:59:41.065101 2747 raft_consensus.cc:2396] T 6518d4af37a1490b88f86b3de2677a5a P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 9 because replica has last-logged OpId of term: 7 index: 25, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 23.
I20240502 07:59:41.065114 2613 raft_consensus.cc:2396] T 6518d4af37a1490b88f86b3de2677a5a P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 9 because replica has last-logged OpId of term: 8 index: 25, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 23.
I20240502 07:59:41.067418 4451 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:41.068509 4320 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:41.064433 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.065428 2946 leader_election.cc:302] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:41.068876 3005 raft_consensus.cc:482] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.068940 3005 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.068995 4853 raft_consensus.cc:2726] T 6518d4af37a1490b88f86b3de2677a5a P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240502 07:59:41.069921 3005 raft_consensus.cc:504] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:41.070659 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.070786 2878 raft_consensus.cc:3032] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Stepping down as leader of term 10
I20240502 07:59:41.070830 2878 raft_consensus.cc:729] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:41.070941 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40a5b1a2b986401ea3166e7a3453cc0e" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:41.071041 2747 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.070991 2878 consensus_queue.cc:260] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:41.071133 2878 raft_consensus.cc:3037] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.071920 2747 raft_consensus.cc:2396] T 40a5b1a2b986401ea3166e7a3453cc0e P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20240502 07:59:41.072156 2878 raft_consensus.cc:2396] T 40a5b1a2b986401ea3166e7a3453cc0e P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20240502 07:59:41.072461 2944 leader_election.cc:302] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:41.072619 4853 raft_consensus.cc:2726] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20240502 07:59:41.073906 3005 leader_election.cc:288] T 40a5b1a2b986401ea3166e7a3453cc0e P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:41.075489 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "18912e0059054584a990be7ccb4a5771"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.075682 3005 raft_consensus.cc:482] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.075796 3005 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.078255 3005 raft_consensus.cc:504] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:41.078770 2727 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:41.081492 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 10 candidate_status { last_received { term: 7 index: 14 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.081650 2878 raft_consensus.cc:3032] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:41.081698 2878 raft_consensus.cc:729] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:41.081861 2878 consensus_queue.cc:260] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 9.15, Last appended by leader: 15, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:41.082031 2878 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.082728 2859 tablet_service.cc:1429] Tablet server has 10 leaders and 0 scanners
I20240502 07:59:41.083169 2878 raft_consensus.cc:2396] T 18912e0059054584a990be7ccb4a5771 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 10 because replica has last-logged OpId of term: 9 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 14.
I20240502 07:59:41.084723 2989 tablet_service.cc:1429] Tablet server has 3 leaders and 0 scanners
I20240502 07:59:41.084797 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18912e0059054584a990be7ccb4a5771" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 10 candidate_status { last_received { term: 7 index: 14 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.084928 2613 raft_consensus.cc:3037] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.086083 2613 raft_consensus.cc:2454] T 18912e0059054584a990be7ccb4a5771 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 10.
I20240502 07:59:41.087280 4713 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:41.087991 2944 leader_election.cc:302] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681
I20240502 07:59:41.089653 4853 raft_consensus.cc:2781] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election won for term 10
I20240502 07:59:41.089757 4853 raft_consensus.cc:686] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:41.089898 4853 consensus_queue.cc:237] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 7.14, Last appended by leader: 14, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:41.092352 2470 catalog_manager.cc:5484] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 9 to 10, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 10 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:41.105533 2597 tablet_service.cc:1429] Tablet server has 2 leaders and 0 scanners
I20240502 07:59:41.106027 4584 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20240502 07:59:41.110118 3005 leader_election.cc:288] T 18912e0059054584a990be7ccb4a5771 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 10 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:41.114030 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f116334237934021a25a6cacae430034"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.114284 3005 raft_consensus.cc:482] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.114423 3005 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.115638 3005 raft_consensus.cc:504] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.116463 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 10 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.116694 2613 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.117683 2613 raft_consensus.cc:2454] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 10.
I20240502 07:59:41.118105 2944 leader_election.cc:302] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:41.118289 4883 raft_consensus.cc:2781] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election won for term 10
I20240502 07:59:41.118355 4883 raft_consensus.cc:686] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:41.118470 4883 consensus_queue.cc:237] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.120556 2470 catalog_manager.cc:5484] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 8 to 10, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 10 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:41.122501 3005 leader_election.cc:288] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 10 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:41.122658 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 10 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:41.122792 2747 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.123813 2747 raft_consensus.cc:2396] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 10.
I20240502 07:59:41.124805 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.124991 3005 raft_consensus.cc:482] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.125088 3005 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.126086 3005 raft_consensus.cc:504] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.126713 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 10 candidate_status { last_received { term: 8 index: 18 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:41.126813 2747 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.127655 2747 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 10.
I20240502 07:59:41.128909 2946 leader_election.cc:302] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:41.129065 4853 raft_consensus.cc:2781] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election won for term 10
I20240502 07:59:41.129133 4853 raft_consensus.cc:686] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:41.129248 4853 consensus_queue.cc:237] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 8.18, Last appended by leader: 18, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.130934 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 10 candidate_status { last_received { term: 8 index: 18 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.131052 2878 raft_consensus.cc:3032] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:41.131091 2878 raft_consensus.cc:729] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:41.131003 2470 catalog_manager.cc:5484] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 9 to 10, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 10 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:41.131213 2878 consensus_queue.cc:260] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 9.19, Last appended by leader: 19, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.131331 2878 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.132441 2878 raft_consensus.cc:2396] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 10 because replica has last-logged OpId of term: 9 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 18.
I20240502 07:59:41.132984 3005 leader_election.cc:288] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 10 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:41.133809 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.133975 3005 raft_consensus.cc:482] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.134083 3005 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.135036 3005 raft_consensus.cc:504] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.135766 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 8 index: 13 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:41.135883 2747 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.136849 2747 raft_consensus.cc:2396] T fe897285bc7f4fcd8ab041aa9874b2ce P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 11 because replica has last-logged OpId of term: 9 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 13.
I20240502 07:59:41.137831 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe897285bc7f4fcd8ab041aa9874b2ce" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 8 index: 13 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.137953 2878 raft_consensus.cc:3037] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.138931 2878 raft_consensus.cc:2454] T fe897285bc7f4fcd8ab041aa9874b2ce P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 11.
I20240502 07:59:41.139387 2944 leader_election.cc:302] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef; no voters: e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:41.139604 4883 raft_consensus.cc:2781] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Leader election won for term 11
I20240502 07:59:41.139675 4883 raft_consensus.cc:686] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [term 11 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:41.139808 4883 consensus_queue.cc:237] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.13, Last appended by leader: 13, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.141230 3005 leader_election.cc:288] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:41.142125 2470 catalog_manager.cc:5484] T fe897285bc7f4fcd8ab041aa9874b2ce P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 9 to 11, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 11 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:41.143877 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d7b590b4226443d4899f3005840fd736"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.144044 3005 raft_consensus.cc:482] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.144155 3005 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.145303 3005 raft_consensus.cc:504] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.146131 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 8 index: 12 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.146255 2878 raft_consensus.cc:3032] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Stepping down as leader of term 10
I20240502 07:59:41.146306 2878 raft_consensus.cc:729] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:41.146454 2878 consensus_queue.cc:260] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.13, Last appended by leader: 13, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.146602 2878 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.147608 2878 raft_consensus.cc:2396] T d7b590b4226443d4899f3005840fd736 P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 11 because replica has last-logged OpId of term: 10 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 12.
I20240502 07:59:41.148595 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7b590b4226443d4899f3005840fd736" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 8 index: 12 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:41.148722 2747 raft_consensus.cc:3037] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.149786 2747 raft_consensus.cc:2454] T d7b590b4226443d4899f3005840fd736 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 11.
I20240502 07:59:41.150199 2946 leader_election.cc:302] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681
I20240502 07:59:41.150408 4853 raft_consensus.cc:2781] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Leader election won for term 11
I20240502 07:59:41.150506 4853 raft_consensus.cc:686] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:41.150648 4853 consensus_queue.cc:237] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.12, Last appended by leader: 12, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.152282 3005 leader_election.cc:288] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:41.152818 2470 catalog_manager.cc:5484] T d7b590b4226443d4899f3005840fd736 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 10 to 11, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 11 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:41.154505 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c67533803b9c496295f4f57e77a51374"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.155309 3005 raft_consensus.cc:482] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.155382 3005 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.156390 3005 raft_consensus.cc:504] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:41.156775 3005 leader_election.cc:288] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 10 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:41.157241 3010 raft_consensus.cc:1261] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 9 index: 18. Preceding OpId from leader: term: 10 index: 19. (index mismatch)
I20240502 07:59:41.158272 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 10 candidate_status { last_received { term: 7 index: 14 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:41.158303 4074 consensus_queue.cc:1035] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 07:59:41.158406 2747 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.159587 2747 raft_consensus.cc:2454] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 10.
I20240502 07:59:41.160063 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.160173 3010 raft_consensus.cc:482] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.160228 3010 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.161089 3010 raft_consensus.cc:504] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.161481 3010 leader_election.cc:288] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 10 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:41.161536 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 10 candidate_status { last_received { term: 7 index: 14 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.161460 2946 leader_election.cc:302] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:41.161689 2878 raft_consensus.cc:3032] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:41.161753 2878 raft_consensus.cc:729] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:41.161788 4883 raft_consensus.cc:2781] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election won for term 10
I20240502 07:59:41.161875 4883 raft_consensus.cc:686] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:41.161913 2878 consensus_queue.cc:260] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 9.15, Last appended by leader: 15, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:41.161998 4883 consensus_queue.cc:237] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 7.14, Last appended by leader: 14, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:41.162055 2878 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.162083 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 10 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.162194 2613 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.162518 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2744610fd94b4a7ca1c7787ecc958bde" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 10 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.162635 2877 raft_consensus.cc:3032] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:41.162686 2877 raft_consensus.cc:729] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:41.162813 2877 consensus_queue.cc:260] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.12, Last appended by leader: 12, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.162952 2877 raft_consensus.cc:3037] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.163089 2878 raft_consensus.cc:2396] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 10 because replica has last-logged OpId of term: 9 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 14.
I20240502 07:59:41.163161 2613 raft_consensus.cc:2396] T 2744610fd94b4a7ca1c7787ecc958bde P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 10 because replica has last-logged OpId of term: 8 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 9.
I20240502 07:59:41.164042 2877 raft_consensus.cc:2396] T 2744610fd94b4a7ca1c7787ecc958bde P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 10 because replica has last-logged OpId of term: 9 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 9.
I20240502 07:59:41.164908 2944 leader_election.cc:302] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:41.165063 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.165153 4883 raft_consensus.cc:2726] T 2744610fd94b4a7ca1c7787ecc958bde P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240502 07:59:41.165171 3005 raft_consensus.cc:482] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.165225 3005 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.166282 3005 raft_consensus.cc:504] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:41.166666 2470 catalog_manager.cc:5484] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 9 to 10, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 10 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:41.167498 2747 raft_consensus.cc:1261] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Refusing update from remote peer c93a6d22180a41a686c4556afd6a3e6b: Log matching property violated. Preceding OpId in replica: term: 9 index: 19. Preceding OpId from leader: term: 10 index: 19. (term mismatch)
I20240502 07:59:41.167501 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.167582 2747 pending_rounds.cc:76] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1: Aborting all ops after (but not including) 18
I20240502 07:59:41.167615 2747 pending_rounds.cc:97] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1: Aborting uncommitted WRITE_OP operation due to leader change: 9.19
I20240502 07:59:41.167614 2613 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Advancing to term 11
W20240502 07:59:41.167752 2747 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Aborted: Op aborted by new leader
I20240502 07:59:41.168381 4074 consensus_queue.cc:1035] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 07:59:41.168545 2613 raft_consensus.cc:2396] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P c93a6d22180a41a686c4556afd6a3e6b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 11 because replica has last-logged OpId of term: 9 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 11.
I20240502 07:59:41.168867 3005 leader_election.cc:288] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:41.169551 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1862c4fc10fc43a8bf7e5d35c2fe47c5" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 2 index: 11 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.169665 2877 raft_consensus.cc:3032] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Stepping down as leader of term 10
I20240502 07:59:41.169711 2877 raft_consensus.cc:729] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:41.169754 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.169854 3005 raft_consensus.cc:482] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.169903 3005 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.169854 2877 consensus_queue.cc:260] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.14, Last appended by leader: 14, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:41.170007 2877 raft_consensus.cc:3037] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.170938 3005 raft_consensus.cc:504] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:41.171059 2877 raft_consensus.cc:2396] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 11 because replica has last-logged OpId of term: 10 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 11.
I20240502 07:59:41.171510 2742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 8 index: 27 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:41.171622 2742 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.171761 2944 leader_election.cc:302] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:41.171962 4853 raft_consensus.cc:2726] T 1862c4fc10fc43a8bf7e5d35c2fe47c5 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20240502 07:59:41.172489 2742 raft_consensus.cc:2454] T 0ecfaed354fe461b8ff36a2ddfa779be P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 11.
I20240502 07:59:41.172772 2946 leader_election.cc:302] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:41.172969 4853 raft_consensus.cc:2781] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Leader election won for term 11
I20240502 07:59:41.173022 3005 leader_election.cc:288] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:41.173046 4853 raft_consensus.cc:686] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [term 11 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:41.173173 4853 consensus_queue.cc:237] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 8.27, Last appended by leader: 27, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
W20240502 07:59:41.174027 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:41.175745 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.175866 3005 raft_consensus.cc:482] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.175932 3005 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.176213 2470 catalog_manager.cc:5484] T 0ecfaed354fe461b8ff36a2ddfa779be P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 10 to 11, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 11 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } }
I20240502 07:59:41.176752 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecfaed354fe461b8ff36a2ddfa779be" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 8 index: 27 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.176867 2877 raft_consensus.cc:3032] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Stepping down as leader of term 10
I20240502 07:59:41.176915 2877 raft_consensus.cc:729] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:41.176954 3005 raft_consensus.cc:504] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.177165 2877 consensus_queue.cc:260] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 10.28, Last appended by leader: 28, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:41.177341 2877 raft_consensus.cc:3037] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.177851 3005 leader_election.cc:288] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:41.177841 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.177940 2878 raft_consensus.cc:3032] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Stepping down as leader of term 10
I20240502 07:59:41.177978 2878 raft_consensus.cc:729] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:41.178081 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee852754568e497897b11ff5a7e2abfd" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.178112 2878 consensus_queue.cc:260] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.178186 2613 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.178231 2878 raft_consensus.cc:3037] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.178310 2877 raft_consensus.cc:2396] T 0ecfaed354fe461b8ff36a2ddfa779be P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 11 because replica has last-logged OpId of term: 10 index: 28, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 27.
I20240502 07:59:41.179126 2878 raft_consensus.cc:2396] T ee852754568e497897b11ff5a7e2abfd P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 8.
I20240502 07:59:41.179152 2613 raft_consensus.cc:2396] T ee852754568e497897b11ff5a7e2abfd P c93a6d22180a41a686c4556afd6a3e6b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 11 because replica has last-logged OpId of term: 9 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 8.
I20240502 07:59:41.179569 2944 leader_election.cc:302] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681, c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:41.179764 4883 raft_consensus.cc:2726] T ee852754568e497897b11ff5a7e2abfd P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20240502 07:59:41.179997 3005 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.180107 3005 raft_consensus.cc:482] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.180172 3005 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.181010 3005 raft_consensus.cc:504] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:41.181361 3005 leader_election.cc:288] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:41.181686 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 10 index: 19 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.181797 2613 raft_consensus.cc:3032] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 10 LEADER]: Stepping down as leader of term 10
I20240502 07:59:41.181792 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fd58a48bc7f4d91b237d8ed1bc0277b" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 10 index: 19 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:41.181847 2613 raft_consensus.cc:729] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:41.181915 2747 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.181978 2613 consensus_queue.cc:260] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 10.19, Last appended by leader: 19, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:41.182163 2613 raft_consensus.cc:3037] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.182868 2747 raft_consensus.cc:2454] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 11.
I20240502 07:59:41.183202 2613 raft_consensus.cc:2454] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 11.
I20240502 07:59:41.183280 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.183390 3010 raft_consensus.cc:482] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.183450 3010 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.183602 2944 leader_election.cc:302] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
W20240502 07:59:41.183754 4835 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45948: Illegal state: Replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:41.183813 4883 raft_consensus.cc:2781] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Leader election won for term 11
I20240502 07:59:41.183888 4883 raft_consensus.cc:686] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [term 11 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:41.183995 4883 consensus_queue.cc:237] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 10.19, Last appended by leader: 19, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:41.184376 3010 raft_consensus.cc:504] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.184808 3010 leader_election.cc:288] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 10 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:41.185151 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 10 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.185288 2878 raft_consensus.cc:3032] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 9 LEADER]: Stepping down as leader of term 9
I20240502 07:59:41.185361 2878 raft_consensus.cc:729] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:41.185484 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a62e14f50e3a41faa564c84e2a991ca1" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 10 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:41.185603 2747 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.185547 2878 consensus_queue.cc:260] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.185708 2878 raft_consensus.cc:3037] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.186491 2747 raft_consensus.cc:2396] T a62e14f50e3a41faa564c84e2a991ca1 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 07:59:41.186635 2878 raft_consensus.cc:2396] T a62e14f50e3a41faa564c84e2a991ca1 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20240502 07:59:41.187155 2944 leader_election.cc:302] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:41.187212 2470 catalog_manager.cc:5484] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 10 to 11, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 11 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:41.187417 4883 raft_consensus.cc:2726] T a62e14f50e3a41faa564c84e2a991ca1 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240502 07:59:41.187824 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.188007 3010 raft_consensus.cc:482] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.188076 3010 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.188942 3010 raft_consensus.cc:504] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.189656 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.189790 2613 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.190001 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410df7ea3ab44e3caa11b2c773d8555a" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.190104 2878 raft_consensus.cc:3032] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Stepping down as leader of term 10
I20240502 07:59:41.190142 2878 raft_consensus.cc:729] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:41.190281 2878 consensus_queue.cc:260] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.190410 2878 raft_consensus.cc:3037] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.190874 2613 raft_consensus.cc:2454] T 410df7ea3ab44e3caa11b2c773d8555a P c93a6d22180a41a686c4556afd6a3e6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 11.
I20240502 07:59:41.191525 2878 raft_consensus.cc:2396] T 410df7ea3ab44e3caa11b2c773d8555a P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
W20240502 07:59:41.191604 2577 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45948: Illegal state: replica c93a6d22180a41a686c4556afd6a3e6b is not leader of this config: current role FOLLOWER
I20240502 07:59:41.191989 3010 leader_election.cc:288] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:41.191974 2944 leader_election.cc:302] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681
I20240502 07:59:41.192159 4883 raft_consensus.cc:2781] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Leader election won for term 11
I20240502 07:59:41.192245 4883 raft_consensus.cc:686] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [term 11 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:41.192389 4883 consensus_queue.cc:237] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.193617 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "811167eb170b49219b6cd22d865af334"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.193763 3010 raft_consensus.cc:482] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.193840 3010 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.194633 3010 raft_consensus.cc:504] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:41.195401 3010 leader_election.cc:288] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Requested vote from peers e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:41.195399 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:41.195515 2747 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.195638 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811167eb170b49219b6cd22d865af334" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.195767 2613 raft_consensus.cc:3032] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 10 LEADER]: Stepping down as leader of term 10
I20240502 07:59:41.195817 2613 raft_consensus.cc:729] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 10 LEADER]: Becoming Follower/Learner. State: Replica: c93a6d22180a41a686c4556afd6a3e6b, State: Running, Role: LEADER
I20240502 07:59:41.196045 2613 consensus_queue.cc:260] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:41.196210 2613 raft_consensus.cc:3037] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.196523 3010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7"
dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
from {username='slave'} at 127.0.0.1:33694
I20240502 07:59:41.196431 2470 catalog_manager.cc:5484] T 410df7ea3ab44e3caa11b2c773d8555a P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 10 to 11, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 11 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:41.196627 3010 raft_consensus.cc:482] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.196645 2747 raft_consensus.cc:2454] T 811167eb170b49219b6cd22d865af334 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 11.
I20240502 07:59:41.196677 3010 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.196938 2946 leader_election.cc:302] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters:
I20240502 07:59:41.197101 4853 raft_consensus.cc:2781] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Leader election won for term 11
I20240502 07:59:41.197175 4853 raft_consensus.cc:686] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:41.197280 4853 consensus_queue.cc:237] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:41.197389 2613 raft_consensus.cc:2396] T 811167eb170b49219b6cd22d865af334 P c93a6d22180a41a686c4556afd6a3e6b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20240502 07:59:41.197659 3010 raft_consensus.cc:504] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.198024 3010 leader_election.cc:288] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 10 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), e0a355821af5483f96aec7e128cc18e1 (127.15.129.130:41015)
I20240502 07:59:41.198715 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 10 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: true dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
I20240502 07:59:41.198740 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fb8f0e2326646b488f6d25a8264e8a7" candidate_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" candidate_term: 10 candidate_status { last_received { term: 7 index: 10 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.198846 2747 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.198846 2613 raft_consensus.cc:3037] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 9 FOLLOWER]: Advancing to term 10
I20240502 07:59:41.199898 2747 raft_consensus.cc:2396] T 3fb8f0e2326646b488f6d25a8264e8a7 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f1e12e7015044fbeb23a7cd6f4023bef for term 10 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 10.
I20240502 07:59:41.200958 2747 raft_consensus.cc:1261] T 3fd58a48bc7f4d91b237d8ed1bc0277b P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 10 index: 19. Preceding OpId from leader: term: 11 index: 21. (index mismatch)
I20240502 07:59:41.201009 2614 raft_consensus.cc:1261] T 3fd58a48bc7f4d91b237d8ed1bc0277b P c93a6d22180a41a686c4556afd6a3e6b [term 11 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 10 index: 19. Preceding OpId from leader: term: 11 index: 21. (index mismatch)
I20240502 07:59:41.201376 4853 consensus_queue.cc:1035] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20240502 07:59:41.201411 2470 catalog_manager.cc:5484] T 811167eb170b49219b6cd22d865af334 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 10 to 11, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 11 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:41.201709 4853 consensus_queue.cc:1035] T 3fd58a48bc7f4d91b237d8ed1bc0277b P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20240502 07:59:41.202809 2613 raft_consensus.cc:2454] T 3fb8f0e2326646b488f6d25a8264e8a7 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1e12e7015044fbeb23a7cd6f4023bef in term 10.
I20240502 07:59:41.203126 2944 leader_election.cc:302] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, f1e12e7015044fbeb23a7cd6f4023bef; no voters: e0a355821af5483f96aec7e128cc18e1
I20240502 07:59:41.203573 4883 raft_consensus.cc:2781] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Leader election won for term 10
I20240502 07:59:41.203660 4883 raft_consensus.cc:686] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 LEADER]: Becoming Leader. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:41.203791 4883 consensus_queue.cc:237] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.207863 2473 catalog_manager.cc:5484] T 3fb8f0e2326646b488f6d25a8264e8a7 P f1e12e7015044fbeb23a7cd6f4023bef reported cstate change: term changed from 8 to 10, leader changed from e0a355821af5483f96aec7e128cc18e1 (127.15.129.130) to f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132). New cstate: current_term: 10 leader_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
W20240502 07:59:41.212414 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
I20240502 07:59:41.215739 2742 raft_consensus.cc:1261] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 9 index: 31. Preceding OpId from leader: term: 11 index: 33. (index mismatch)
I20240502 07:59:41.215809 2614 raft_consensus.cc:1261] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 11 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 10 index: 32. Preceding OpId from leader: term: 11 index: 33. (index mismatch)
I20240502 07:59:41.216140 4853 consensus_queue.cc:1035] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20240502 07:59:41.216468 4883 consensus_queue.cc:1035] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20240502 07:59:41.216569 4883 consensus_queue.cc:1225] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Peer c93a6d22180a41a686c4556afd6a3e6b log is divergent from this leader: its last log entry 10.32 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 31
I20240502 07:59:41.217193 2614 pending_rounds.cc:76] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b: Aborting all ops after (but not including) 31
I20240502 07:59:41.217242 2614 pending_rounds.cc:97] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b: Aborting uncommitted NO_OP operation due to leader change: 10.32
I20240502 07:59:41.217270 2614 raft_consensus.cc:2866] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20240502 07:59:41.222726 2708 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60702: Illegal state: replica e0a355821af5483f96aec7e128cc18e1 is not leader of this config: current role FOLLOWER
W20240502 07:59:41.228101 2836 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34834: Illegal state: replica 606c439025f24e53a2feebeaf68c1681 is not leader of this config: current role FOLLOWER
I20240502 07:59:41.237016 2742 raft_consensus.cc:1261] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 8 index: 18. Preceding OpId from leader: term: 10 index: 20. (index mismatch)
I20240502 07:59:41.237039 2878 raft_consensus.cc:1261] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Refusing update from remote peer f1e12e7015044fbeb23a7cd6f4023bef: Log matching property violated. Preceding OpId in replica: term: 9 index: 19. Preceding OpId from leader: term: 10 index: 20. (index mismatch)
I20240502 07:59:41.237651 5154 consensus_queue.cc:1035] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 07:59:41.237776 5154 consensus_queue.cc:1225] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Peer 606c439025f24e53a2feebeaf68c1681 log is divergent from this leader: its last log entry 9.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20240502 07:59:41.238032 5154 consensus_queue.cc:1035] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20240502 07:59:41.240289 2878 pending_rounds.cc:76] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681: Aborting all ops after (but not including) 18
I20240502 07:59:41.240357 2878 pending_rounds.cc:97] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681: Aborting uncommitted NO_OP operation due to leader change: 9.19
I20240502 07:59:41.240427 2878 raft_consensus.cc:2866] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240502 07:59:41.243357 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "512e1322925c415f8891dfe6621160e1"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:41.243482 2742 raft_consensus.cc:482] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.243533 2742 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Advancing to term 12
I20240502 07:59:41.246407 5155 mvcc.cc:205] Tried to move back new op lower bound from 7023152255942721536 to 7023152255506583552. Current Snapshot: MvccSnapshot[applied={T|T < 7023152255942721536}]
I20240502 07:59:41.257927 2742 raft_consensus.cc:504] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.258510 2742 leader_election.cc:288] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 12 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:41.258790 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 12 candidate_status { last_received { term: 9 index: 13 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.258906 2877 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Advancing to term 12
I20240502 07:59:41.259773 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c67533803b9c496295f4f57e77a51374"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:41.259886 2742 raft_consensus.cc:482] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.259873 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "512e1322925c415f8891dfe6621160e1" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 12 candidate_status { last_received { term: 9 index: 13 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:41.259948 2742 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.260004 3010 raft_consensus.cc:3037] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Advancing to term 12
I20240502 07:59:41.260020 2877 raft_consensus.cc:2396] T 512e1322925c415f8891dfe6621160e1 P 606c439025f24e53a2feebeaf68c1681 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 12 because replica has last-logged OpId of term: 10 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 13.
I20240502 07:59:41.260962 2742 raft_consensus.cc:504] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:41.261076 3010 raft_consensus.cc:2454] T 512e1322925c415f8891dfe6621160e1 P f1e12e7015044fbeb23a7cd6f4023bef [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 12.
I20240502 07:59:41.261276 2742 leader_election.cc:288] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 11 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:41.261642 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 11 candidate_status { last_received { term: 6 index: 13 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:41.261762 3010 raft_consensus.cc:3032] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 LEADER]: Stepping down as leader of term 10
I20240502 07:59:41.261803 3010 raft_consensus.cc:729] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:41.261847 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c67533803b9c496295f4f57e77a51374" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 11 candidate_status { last_received { term: 6 index: 13 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.261955 2877 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.261962 3010 consensus_queue.cc:260] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 10.15, Last appended by leader: 15, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:41.262125 3010 raft_consensus.cc:3037] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.262804 2877 raft_consensus.cc:2396] T c67533803b9c496295f4f57e77a51374 P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 11 because replica has last-logged OpId of term: 9 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 13.
I20240502 07:59:41.263201 3010 raft_consensus.cc:2396] T c67533803b9c496295f4f57e77a51374 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 11 because replica has last-logged OpId of term: 10 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 13.
I20240502 07:59:41.263594 2681 leader_election.cc:302] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1, f1e12e7015044fbeb23a7cd6f4023bef; no voters: 606c439025f24e53a2feebeaf68c1681
I20240502 07:59:41.263826 4193 raft_consensus.cc:2781] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 12 FOLLOWER]: Leader election won for term 12
I20240502 07:59:41.263926 2681 leader_election.cc:302] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0a355821af5483f96aec7e128cc18e1; no voters: 606c439025f24e53a2feebeaf68c1681, f1e12e7015044fbeb23a7cd6f4023bef
I20240502 07:59:41.263937 4193 raft_consensus.cc:686] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [term 12 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:41.264398 4193 consensus_queue.cc:237] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 9.13, Last appended by leader: 13, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.265378 4906 raft_consensus.cc:2726] T c67533803b9c496295f4f57e77a51374 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20240502 07:59:41.265609 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:41.265709 2742 raft_consensus.cc:482] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.265763 2742 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.266510 2742 raft_consensus.cc:504] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:41.266427 2473 catalog_manager.cc:5484] T 512e1322925c415f8891dfe6621160e1 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 10 to 12, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 12 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:41.266817 2742 leader_election.cc:288] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 11 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:41.267299 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 11 candidate_status { last_received { term: 8 index: 12 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.267410 2877 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.268136 2877 raft_consensus.cc:2454] T 001fcdef92e8443baf7ac443a1b4c57a P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 11.
I20240502 07:59:41.268714 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:41.268816 2742 raft_consensus.cc:482] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.268873 2742 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.268889 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001fcdef92e8443baf7ac443a1b4c57a" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 11 candidate_status { last_received { term: 8 index: 12 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.269032 2614 raft_consensus.cc:3037] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.269192 2682 leader_election.cc:302] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:41.269755 2742 raft_consensus.cc:504] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.270066 2742 leader_election.cc:288] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 11 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:41.270130 2614 raft_consensus.cc:2396] T 001fcdef92e8443baf7ac443a1b4c57a P c93a6d22180a41a686c4556afd6a3e6b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 11 because replica has last-logged OpId of term: 9 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 12.
I20240502 07:59:41.270349 4193 raft_consensus.cc:2781] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Leader election won for term 11
I20240502 07:59:41.270437 4193 raft_consensus.cc:686] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [term 11 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:41.270592 4193 consensus_queue.cc:237] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.12, Last appended by leader: 12, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } }
I20240502 07:59:41.270880 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.270978 2877 raft_consensus.cc:3032] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Stepping down as leader of term 10
I20240502 07:59:41.271020 2877 raft_consensus.cc:729] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 606c439025f24e53a2feebeaf68c1681, State: Running, Role: LEADER
I20240502 07:59:41.271160 2877 consensus_queue.cc:260] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.271291 2877 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.272337 2877 raft_consensus.cc:2396] T 1afc3e666fcb4b1296f0d553f2835e0d P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20240502 07:59:41.273583 2742 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f116334237934021a25a6cacae430034"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:41.273696 2742 raft_consensus.cc:482] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.273567 2473 catalog_manager.cc:5484] T 001fcdef92e8443baf7ac443a1b4c57a P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 9 to 11, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 11 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } }
I20240502 07:59:41.273746 2742 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.274822 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1afc3e666fcb4b1296f0d553f2835e0d" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.274931 2614 raft_consensus.cc:3037] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.274617 2742 raft_consensus.cc:504] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.276822 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.276932 2613 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.276773 2742 leader_election.cc:288] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 11 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:41.277545 2614 raft_consensus.cc:2454] T 1afc3e666fcb4b1296f0d553f2835e0d P c93a6d22180a41a686c4556afd6a3e6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 11.
I20240502 07:59:41.277796 2682 leader_election.cc:302] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters: 606c439025f24e53a2feebeaf68c1681
I20240502 07:59:41.278002 4193 raft_consensus.cc:2781] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Leader election won for term 11
I20240502 07:59:41.278076 4193 raft_consensus.cc:686] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [term 11 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:41.278213 4193 consensus_queue.cc:237] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.278249 2613 raft_consensus.cc:2454] T f116334237934021a25a6cacae430034 P c93a6d22180a41a686c4556afd6a3e6b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 11.
I20240502 07:59:41.278712 2682 leader_election.cc:302] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:41.278841 4906 raft_consensus.cc:2781] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Leader election won for term 11
I20240502 07:59:41.278909 4906 raft_consensus.cc:686] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [term 11 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:41.279011 4906 consensus_queue.cc:237] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.279301 3010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f116334237934021a25a6cacae430034" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:41.279419 3010 raft_consensus.cc:3032] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 LEADER]: Stepping down as leader of term 10
I20240502 07:59:41.279460 3010 raft_consensus.cc:729] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:41.279629 3010 consensus_queue.cc:260] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.279769 3010 raft_consensus.cc:3037] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.281144 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:41.281277 2747 raft_consensus.cc:482] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.281356 2747 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.282348 2747 raft_consensus.cc:504] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.282730 2747 leader_election.cc:288] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 11 election: Requested vote from peers 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417), f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815)
I20240502 07:59:41.283085 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 11 candidate_status { last_received { term: 10 index: 23 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.283205 2878 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.284118 3010 raft_consensus.cc:2396] T f116334237934021a25a6cacae430034 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20240502 07:59:41.284214 2878 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 11.
I20240502 07:59:41.284341 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f832b61170a4f2d95f3494c23e67e55" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 11 candidate_status { last_received { term: 10 index: 23 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:41.284437 3005 raft_consensus.cc:3032] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 LEADER]: Stepping down as leader of term 10
I20240502 07:59:41.284474 3005 raft_consensus.cc:729] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:41.284540 2682 leader_election.cc:302] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:41.284623 3005 consensus_queue.cc:260] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 10.23, Last appended by leader: 23, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.284758 3005 raft_consensus.cc:3037] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.284760 4193 raft_consensus.cc:2781] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Leader election won for term 11
I20240502 07:59:41.284835 4193 raft_consensus.cc:686] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [term 11 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:41.284770 2473 catalog_manager.cc:5484] T 1afc3e666fcb4b1296f0d553f2835e0d P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 10 to 11, leader changed from 606c439025f24e53a2feebeaf68c1681 (127.15.129.131) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 11 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:41.284933 4193 consensus_queue.cc:237] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 10.23, Last appended by leader: 23, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } }
I20240502 07:59:41.285001 2473 catalog_manager.cc:5484] T f116334237934021a25a6cacae430034 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 10 to 11, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 11 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:41.285732 3005 raft_consensus.cc:2454] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 11.
I20240502 07:59:41.286177 2877 raft_consensus.cc:1226] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Rejecting Update request from peer f1e12e7015044fbeb23a7cd6f4023bef for earlier term 10. Current term is 11. Ops: []
I20240502 07:59:41.287199 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:41.287372 2747 raft_consensus.cc:482] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.287532 2747 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Advancing to term 12
I20240502 07:59:41.288846 2747 raft_consensus.cc:504] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.288822 2473 catalog_manager.cc:5484] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 10 to 11, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 11 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } }
I20240502 07:59:41.289350 2747 leader_election.cc:288] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 12 election: Requested vote from peers f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132:37815), c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183)
I20240502 07:59:41.289570 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 12 candidate_status { last_received { term: 11 index: 34 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.289705 2613 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 11 FOLLOWER]: Advancing to term 12
I20240502 07:59:41.290042 3005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5ce1f24891642c68ef1cf85f15158a5" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 12 candidate_status { last_received { term: 11 index: 34 } } ignore_live_leader: true dest_uuid: "f1e12e7015044fbeb23a7cd6f4023bef"
I20240502 07:59:41.290159 3005 raft_consensus.cc:3032] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 LEADER]: Stepping down as leader of term 11
I20240502 07:59:41.290202 3005 raft_consensus.cc:729] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f1e12e7015044fbeb23a7cd6f4023bef, State: Running, Role: LEADER
I20240502 07:59:41.290335 3005 consensus_queue.cc:260] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 11.34, Last appended by leader: 34, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.290518 3005 raft_consensus.cc:3037] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Advancing to term 12
I20240502 07:59:41.290745 2613 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P c93a6d22180a41a686c4556afd6a3e6b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 12.
I20240502 07:59:41.291097 2682 leader_election.cc:302] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c93a6d22180a41a686c4556afd6a3e6b, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:41.291285 4906 raft_consensus.cc:2781] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 12 FOLLOWER]: Leader election won for term 12
I20240502 07:59:41.291363 4906 raft_consensus.cc:686] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [term 12 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:41.291467 3005 raft_consensus.cc:2454] T a5ce1f24891642c68ef1cf85f15158a5 P f1e12e7015044fbeb23a7cd6f4023bef [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 12.
I20240502 07:59:41.291481 4906 consensus_queue.cc:237] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 34, Committed index: 34, Last appended: 11.34, Last appended by leader: 34, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.291735 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:41.291833 2747 raft_consensus.cc:482] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.291882 2747 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.292778 2747 raft_consensus.cc:504] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.293437 2613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 11 candidate_status { last_received { term: 8 index: 12 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.293376 2468 catalog_manager.cc:5484] T a5ce1f24891642c68ef1cf85f15158a5 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 11 to 12, leader changed from f1e12e7015044fbeb23a7cd6f4023bef (127.15.129.132) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 12 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1e12e7015044fbeb23a7cd6f4023bef" member_type: VOTER last_known_addr { host: "127.15.129.132" port: 37815 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
I20240502 07:59:41.293534 2613 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.294438 2613 raft_consensus.cc:2396] T 37b0b5493f8748338d5c373eaee489d4 P c93a6d22180a41a686c4556afd6a3e6b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e0a355821af5483f96aec7e128cc18e1 for term 11 because replica has last-logged OpId of term: 9 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 12.
I20240502 07:59:41.294672 2747 leader_election.cc:288] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 11 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:41.295483 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37b0b5493f8748338d5c373eaee489d4" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 11 candidate_status { last_received { term: 8 index: 12 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.295648 2877 raft_consensus.cc:3037] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.296705 2877 raft_consensus.cc:2454] T 37b0b5493f8748338d5c373eaee489d4 P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 11.
I20240502 07:59:41.296978 2682 leader_election.cc:302] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters: c93a6d22180a41a686c4556afd6a3e6b
I20240502 07:59:41.297286 4193 raft_consensus.cc:2781] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Leader election won for term 11
I20240502 07:59:41.297402 4193 raft_consensus.cc:686] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [term 11 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:41.297530 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:41.297520 4193 consensus_queue.cc:237] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 8.12, Last appended by leader: 12, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.297837 2747 raft_consensus.cc:482] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.297905 2747 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.299119 2473 catalog_manager.cc:5484] T 37b0b5493f8748338d5c373eaee489d4 P e0a355821af5483f96aec7e128cc18e1 reported cstate change: term changed from 9 to 11, leader changed from c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129) to e0a355821af5483f96aec7e128cc18e1 (127.15.129.130). New cstate: current_term: 11 leader_uuid: "e0a355821af5483f96aec7e128cc18e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } }
W20240502 07:59:41.301494 2970 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33714: Illegal state: replica f1e12e7015044fbeb23a7cd6f4023bef is not leader of this config: current role FOLLOWER
I20240502 07:59:41.301194 2747 raft_consensus.cc:504] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:41.302716 2747 leader_election.cc:288] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 11 election: Requested vote from peers c93a6d22180a41a686c4556afd6a3e6b (127.15.129.129:46183), 606c439025f24e53a2feebeaf68c1681 (127.15.129.131:35417)
I20240502 07:59:41.303256 2747 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880"
dest_uuid: "e0a355821af5483f96aec7e128cc18e1"
from {username='slave'} at 127.0.0.1:60732
I20240502 07:59:41.303362 2747 raft_consensus.cc:482] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20240502 07:59:41.303423 2747 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.303501 2877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 11 candidate_status { last_received { term: 8 index: 20 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.303588 2614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65116bced0d2437fa9a49e0391d3d4ac" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 11 candidate_status { last_received { term: 8 index: 20 } } ignore_live_leader: true dest_uuid: "c93a6d22180a41a686c4556afd6a3e6b"
I20240502 07:59:41.303668 2877 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.303719 2614 raft_consensus.cc:3037] T 65116bced0d2437fa9a49e0391d3d4ac P c93a6d22180a41a686c4556afd6a3e6b [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.304317 2747 raft_consensus.cc:504] T 4648d978bccc4a65b30342ceaa7cf880 P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } } peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } }
I20240502 07:59:41.304862 2877 raft_consensus.cc:2454] T 65116bced0d2437fa9a49e0391d3d4ac P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a355821af5483f96aec7e128cc18e1 in term 11.
I20240502 07:59:41.304945 2878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4648d978bccc4a65b30342ceaa7cf880" candidate_uuid: "e0a355821af5483f96aec7e128cc18e1" candidate_term: 11 candidate_status { last_received { term: 8 index: 11 } } ignore_live_leader: true dest_uuid: "606c439025f24e53a2feebeaf68c1681"
I20240502 07:59:41.305042 2878 raft_consensus.cc:3037] T 4648d978bccc4a65b30342ceaa7cf880 P 606c439025f24e53a2feebeaf68c1681 [term 10 FOLLOWER]: Advancing to term 11
I20240502 07:59:41.305264 2682 leader_election.cc:302] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 606c439025f24e53a2feebeaf68c1681, e0a355821af5483f96aec7e128cc18e1; no voters:
I20240502 07:59:41.305491 4193 raft_consensus.cc:2781] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 11 FOLLOWER]: Leader election won for term 11
I20240502 07:59:41.305564 4193 raft_consensus.cc:686] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [term 11 LEADER]: Becoming Leader. State: Replica: e0a355821af5483f96aec7e128cc18e1, State: Running, Role: LEADER
I20240502 07:59:41.305680 4193 consensus_queue.cc:237] T 65116bced0d2437fa9a49e0391d3d4ac P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 8.20, Last appended by leader: 20, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c93a6d22180a41a686c4556afd6a3e6b" member_type: VOTER last_known_addr { host: "127.15.129.129" port: 46183 } } peers { permanent_uuid: "e0a355821af5483f96aec7e128cc18e1" member_type: VOTER last_known_addr { host: "127.15.129.130" port: 41015 } } peers { permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131" port: 35417 } }
I20240502 07:59:41.307555 2877 raft_consensus.cc:1261] T 1f832b61170a4f2d95f3494c23e67e55 P 606c439025f24e53a2feebeaf68c1681 [term 11 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 10 index: 23. Preceding OpId from leader: term: 11 index: 25. (index mismatch)
I20240502 07:59:41.307823 3005 raft_consensus.cc:1261] T 1f832b61170a4f2d95f3494c23e67e55 P f1e12e7015044fbeb23a7cd6f4023bef [term 11 FOLLOWER]: Refusing update from remote peer e0a355821af5483f96aec7e128cc18e1: Log matching property violated. Preceding OpId in replica: term: 10 index: 23. Preceding OpId from leader: term: 11 index: 25. (index mismatch)
I20240502 07:59:41.307967 4906 consensus_queue.cc:1035] T 1f832b61170a4f2d95f3494c23e67e55 P e0a355821af5483f96aec7e128cc18e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "606c439025f24e53a2feebeaf68c1681" member_type: VOTER last_known_addr { host: "127.15.129.131"